Fail Over Warning: Rebalance recommended, some data does not have the desired number of replicas

990 views
Skip to first unread message

Lasse Schou

unread,
May 24, 2013, 3:38:04 AM5/24/13
to couc...@googlegroups.com
Hi all,

After changing the number of replicas from 1 to 2 through the unsupported /diag/eval call, proposed by Aliaksey (see below), I keep getting this message (see subject) no matter how many times I rebalance. I've even tried replacing all servers one by one, but the message has stuck. What I did was increasing the replicas to 2. I could finish rebalance after that, so I set the replicas down to one again, after which the rebalance worked fine again. Except from this message:

Fail Over Warning: Rebalance recommended, some data does not have the desired number of replicas!

This was the call I used to increase and descrease replicas.

wget -O- --user=Administrator --password=xxxxx --post-data='ReplicasNum = 2, BucketName = "default", {ok, Config} = ns_bucket:get_bucket(BucketName), Map = [lists:sublist(Chain ++ [undefined, undefined, undefined, undefined], ReplicasNum+1) || Chain <- proplists:get_value(map, Config)], ns_bucket:update_bucket_props(BucketName, [{map, Map}, {num_replicas,ReplicasNum}]).' http://127.0.0.1:8091/diag/eval

Any idea how I get rid of the message?

Lasse

Aliaksey Kandratsenka

unread,
May 24, 2013, 3:23:09 PM5/24/13
to couc...@googlegroups.com
Most likely this happens because you have 2 nodes and full replication (master + 2 replicas) requires 3.

If not please file a bug. That would be something new.

Lasse Schou

unread,
May 24, 2013, 3:44:39 PM5/24/13
to couc...@googlegroups.com
Hi. I have 5 nodes. And as I wrote I didn't manage to rebalance with 2 replicas so I switched back to 1 replica using the same call. And after many rebalances I get the mentioned message. Any idea? If not, can you explain how I file a bug?
--
You received this message because you are subscribed to the Google Groups "Couchbase" group.
To unsubscribe from this group and stop receiving emails from it, send an email to couchbase+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 


--
Sent from my phone.

Aliaksey Kandratsenka

unread,
May 24, 2013, 4:30:58 PM5/24/13
to couc...@googlegroups.com
On Fri, May 24, 2013 at 12:44 PM, Lasse Schou <lasse...@gmail.com> wrote:
Hi. I have 5 nodes. And as I wrote I didn't manage to rebalance with 2 replicas so I switched back to 1 replica using the same call. And after many rebalances I get the mentioned message. Any idea? If not, can you explain how I file a bug?

Please file bug here: http://www.couchbase.com/issues/secure/Dashboard.jspa and assign it to me.

and attach output of cbcollect_info from at least one node.

Particularly I'll need your vbucket map but logs may also hint whats up. cbcollect_info has all of that.

With that information I expect I'll be able to understand whats wrong and how to proceed.

Tugdual Grall

unread,
May 28, 2013, 3:57:45 AM5/28/13
to couc...@googlegroups.com
Hello Lasse,

Have you filed a bug? if yes could you please put it this thread this will allow the community to follow the discussion.

If not do you still have the issue?

regards
Tug
To unsubscribe from this group and stop receiving emails from it, send an email to couchbase+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.
 
 

Lasse Schou

unread,
May 28, 2013, 4:07:12 AM5/28/13
to couc...@googlegroups.com
Still waiting to get access to the JIRA system. Sent out a request.


2013/5/28 Tugdual Grall <tug...@gmail.com>
To unsubscribe from this group and stop receiving emails from it, send an email to couchbase+...@googlegroups.com.

Kenny Garland

unread,
Jun 6, 2013, 4:37:29 PM6/6/13
to couc...@googlegroups.com
I'm also seeing this problem on our cluster.

5 nodes
7 couch buckets
1 memcache bucket

Only 3 of the buckets are using replicas. One has 1 replica, another has 2 and the final one has 3 replicas.

These are all Amazon EC2 m1.medium instances. 4 of the instances are running CB 2.0.0 and one is running 2.0.1

 - Ken

Aliaksey Kandratsenka

unread,
Jun 6, 2013, 4:42:43 PM6/6/13
to couc...@googlegroups.com
On Thu, Jun 6, 2013 at 1:37 PM, Kenny Garland <garl...@gmail.com> wrote:
I'm also seeing this problem on our cluster.

5 nodes
7 couch buckets
1 memcache bucket

Only 3 of the buckets are using replicas. One has 1 replica, another has 2 and the final one has 3 replicas.

These are all Amazon EC2 m1.medium instances. 4 of the instances are running CB 2.0.0 and one is running 2.0.1


It safe to ignore this warning.

Lasse Schou

unread,
Jun 6, 2013, 4:45:42 PM6/6/13
to couc...@googlegroups.com
I reduced the bucket with 3 replicas to 2 replicas, and it fixed the issue.


2013/6/6 Aliaksey Kandratsenka <alkond...@gmail.com>

--

Kenny Garland

unread,
Jun 10, 2013, 6:44:52 AM6/10/13
to couc...@googlegroups.com
To change the replicas you have to delete the bucket and re-create it, does that mean it is possible something else fixed the issue when the delete occurred?

We have been using 3 replicas on this bucket for a long time and only when I added a new server to the cluster did this error appear.

Aliaksey Kandratsenka

unread,
Jun 10, 2013, 12:58:41 PM6/10/13
to couc...@googlegroups.com
On Mon, Jun 10, 2013 at 3:44 AM, Kenny Garland <garl...@gmail.com> wrote:
To change the replicas you have to delete the bucket and re-create it, does that mean it is possible something else fixed the issue when the delete occurred?

We have been using 3 replicas on this bucket for a long time and only when I added a new server to the cluster did this error appear.

Ticket I referred to explains it. This only happens for exactly 5 nodes and exactly 3 replicas.

Rob Pecor

unread,
Jan 14, 2015, 11:32:15 AM1/14/15
to couc...@googlegroups.com
I am getting this using 3 nodes, 1 replica.  This is on a fresh install with only the default bucket set to 200MB, using Couchbase Bucket Type.
Using 3.0.2-1603 Enterprise Edition (build-1603)

Warren Lindsey

unread,
Jan 14, 2015, 2:56:57 PM1/14/15
to couc...@googlegroups.com
Are you using the rack awareness feature?

--
You received this message because you are subscribed to the Google Groups "Couchbase" group.
To unsubscribe from this group and stop receiving emails from it, send an email to couchbase+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Reply all
Reply to author
Forward
0 new messages