Max buckets on Couchbase 2.0

[Ken Nakagama]

It seemed best to ask this in public, but I have been asked by others using Couchbase.

The docs are confusing and ironic sometimes.

As we are developing a public benchmark for Couchbase, versus other things.

I want to know

1) What are the max buckets for min performance.

2) What is today, the suggested for 2.0

Answering a question with a question is normally okay, but without any answers its not

It tends to tell people your time is worth more than others and it probably is)

Cheers

[Hans Kristian Grani]

http://www.couchbase.com/docs/couchbase-manual-2.0/couchbase-server-limits.html

Max 10 buckets per Cluster

[Jeff Minard]

That is extraordinarily small -- I had expected the number to be closer to "as many as you have file descriptors/ram for".

Can any one illuminate the restriction?

[Chad Kouse]

/me wagers a guess: creating many buckets would make the management channel very busy.

-- chad

[scalabl3]

Correct. It's a combination of management sockets/communication, threads, stats gathering, file descriptors and RAM. After 10-15 buckets… the CPU overhead grows significantly. However, we are working on some new ideas that will change how some of that works, not likely to be completed in the short term however, no dates on when. Just things in the works, multi-tenant solutions is a priority for us which involves a large number of buckets.

[Ken Nakagama]

Thanks everyone

just wanted to check beyond what I had already found.

2.0 is pretty super though

[hajo.kli...@gmail.com]

is there any change since this topic was created?

[Aliaksey Kandratsenka]

On Fri, Jul 26, 2013 at 11:08 AM, <hajo.kli...@gmail.com> wrote:

is there any change since this topic was created?

Some improvements were done. Couchbase Server should now handle larger number of buckets more efficiently. In some experiments folks could run few hundreds of buckets.

But we still do not recommend it and do not officially support. For best performance don't do many buckets.

[Piyush Gupta]

what is the configuration of the nodes (CPU/RAM/HDD) on which such tests/experiments were done?  could you share that please?

[Aliaksey Kandratsenka]

On Wed, Jul 31, 2013 at 11:34 AM, Piyush Gupta <p77...@gmail.com> wrote:

what is the configuration of the nodes (CPU/RAM/HDD) on which such tests/experiments were done?  could you share that please?

I don't know exactly. It was some larger EC2 instance type. Also note that when  I said "could run few hundreds" it doesn't mean running those few hundreds was efficient.

[Piyush Gupta]

hmmm... i get that! :-)  would be interesting to try it out though.  and see how good (or bad) it is.

--
You received this message because you are subscribed to the Google Groups "Couchbase" group.
To unsubscribe from this group and stop receiving emails from it, send an email to couchbase+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
"I am not lazy, I just rest before I get tired"

                                         - dust graffiti

[hajo.kli...@gmail.com]

i will ask again, since couchbase 2.2 was released :) any changes?

[Aliaksey Kandratsenka]

On Mon, Oct 21, 2013 at 8:33 AM, <hajo.kli...@gmail.com> wrote:

i will ask again, since couchbase 2.2 was released :) any changes?

As pointed out above expect just relatively minor improvements. People could run tons of buckets with special tuning. I.e. lowering number of vbuckets. But there's still pretty significant overhead.

[Vitaly Elyashev]

Hi,

How I can lower number of vbuckets?

[Jim Walker]

A simple way you can change the number of vbuckets is by using an environment variable. I've done this for a variety of performance tests I did sometime last year and it works well providing you follow my caveats.

For example in bash, set the value then start the node or for sanity explicitly add the value into the init.d script.

> export COUCHBASE_NUM_VBUCKETS=512

> /etc/init.d/couchbase start

The lowest you can go is a value of 4, to go lower requires changing another config variable which I cannot assert won't blow up if changed as I never needed to go lower.

Caveats:

Changing this variable is not supported by Couchbase! So only do this on non-production systems for your own experimentation.
I recommend using a new cluster. Don't change this on a running or existing yet dormant cluster.
Do this before creating any buckets. Note that existing buckets would lose data because some of the existing VB files become out-of-range.
It's safest to set this on each nodes before you start the cluster, I've never asserted if the cluster manager will propagate this variable to each node and at least this allowed me to destroy the cluster and restart from any node.

[Jim Walker]

If you want to go larger, you'll find the Couchbase wire-protocol reserves 16-bits for the vbucket ID, so max uint16 is your upper limit.