Best Configuration for a Production Enviorment - project-voldemort

The old Google Groups will be going away soon, but your browser is incompatible with the new version.
Message from discussion Best Configuration for a Production Enviorment
Received-SPF: pass (google.com: domain of lslus...@gmail.com designates 209.85.214.182 as permitted sender) client-ip=209.85.214.182;
Received-SPF: pass (google.com: domain of lslus...@gmail.com designates 10.182.174.101 as permitted sender) client-ip=10.182.174.101;
MIME-Version: 1.0
In-Reply-To: <CAMu2xmNk0PD28PT7AdyoGN7ALMS_syk_gvCc8Gd6iH6MOd4...@mail.gmail.com>
References: <33290347.385.1330625030419.JavaMail.geo-discussion-forums@vbva11>
	<CAMu2xmNk0PD28PT7AdyoGN7ALMS_syk_gvCc8Gd6iH6MOd4...@mail.gmail.com>
Date: Thu, 1 Mar 2012 11:14:04 -0800
Message-ID: <CAESZ+_8TmAoOK-9TToLG-DDnew-sYow21sASWCN_RFdsiW5...@mail.gmail.com>
Subject: Re: [project-voldemort] Best Configuration for a Production Enviorment
From: Liam Slusser <lslus...@gmail.com>
To: project-voldemort@googlegroups.com
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Have you looked into using SSD's for storage?  We have around 5m keys
~18k per key using BDB and had performance issues with a 8 node
cluster setup.  I replaced our cluster with 4 servers with 16g ram and
each with a raid10 SSD storage and haven't looked back.

liam

On Thu, Mar 1, 2012 at 10:49 AM, Mickey Hsieh <micke...@gmail.com> wrote:
> Given 3m (rows) * 1mb (data) * 2 (replica) /3 =A0(node) =3D 2 TB /per nod=
e. How
> did you get 60GB =A0data size per node? =A0Am I missing some thing?
>
> I can share with you my experience. We had using Voldemort Project for mo=
re
> 2 years neither BDB and MySql can not meet our requirement. We had develo=
ped
> our storage plug-in open source CacheStore
> (http://code.google.com/p/cachestore) =A0to handle the load and performan=
ce.
>
> Here use case:
> we have 600m (rows) * 1 kb (data) * 2 (replica) / 20 nodes =3D 60 GB / pe=
r
> node
> 1500 tps random read/write ( 500 read + 1000 write ) / =A0per seconds
> on client side is around 5 - 7 ms/tps . on serve side is less than 1 ms/t=
ps
>
> If you are interested, I can chat and understand your requirement.
>
> Mickey
>
> On Thu, Mar 1, 2012 at 10:03 AM, ctasada <ctas...@gmail.com> wrote:
>>
>> Hi everyone,
>>
>> Since some weeks ago I'm involved in a project that's using
>> project-voldemort as a backend. We're having some performance issues due=
 to
>> an increment in the system load.
>>
>> For what I've been reading in the group and checking the code I've some
>> ideas, but I wanted to ask for your experience in the matter.
>>
>> Current Configuration:
>> 3 Nodes with replication factor 2, Hinted Handoff enabled and Read/write=
s
>> 1. We're working with around 3 million keys and values around 1MB. The n=
odes
>> are storing more than 60GB BDB data files (I need to check the BDB clean=
er
>> configuration since this seems quite big). The servers are 8CPU Linux 64=
b
>> with 44GB memory, using 22GB in the JVM and 10GB BDB cache (which I know=
n is
>> really small right now)
>>
>> I'm seeing BDB locks and high read/write loads in disk. My impression is
>> that's caused by the values size and lack of enough cache for the BDB fi=
les,
>> so that's what i'm planning to do:
>> - Rebalance the partitions in more nodes.
>> - Modify my code to decrease the value size to something smaller, probab=
ly
>> between 100KB and 300KB (increasing the number of keys)
>>
>> What I'm expecting to get are smaller BDB files that can fit better in
>> cache and smaller values that can be read/write to disk faster.
>>
>> What I'm not really sure about is:
>> - What's the best balance between JVM memory, BDB Cache and BDB size?
>> - Can you think of any problem running 2 Voldemort server instances in t=
he
>> same server? would be better to use different VM,=A0when there're in the=
 same
>> physical server?
>> - Is there any kind of hardware recommendation or preferred configuratio=
n
>> when working with Voldemort?
>>
>> Your help is really appreciated.
>>
>> Best Regards,
>> Carlos.
>>
>> --
>> You received this message because you are subscribed to the Google Group=
s
>> "project-voldemort" group.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msg/project-voldemort/-/mEfpkMzAsQkJ.
>> To post to this group, send email to project-voldemort@googlegroups.com.
>> To unsubscribe from this group, send email to
>> project-voldemort+unsubscribe@googlegroups.com.
>> For more options, visit this group at
>> http://groups.google.com/group/project-voldemort?hl=3Den.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "project-voldemort" group.
> To post to this group, send email to project-voldemort@googlegroups.com.
> To unsubscribe from this group, send email to
> project-voldemort+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/project-voldemort?hl=3Den.
Create a group - Google Groups - Google Home - Terms of Service - Privacy Policy
Account Options
