The list is a bit quiet - I hope somebody can shine light on this - even if I'm totally off track here.
I have a multi-threaded server that acts as a client to Voldemort, The threads are more like actors/processes, in that they share nothing except the Voldemort clients. These are obtained from a singleton, and are shared across threads.
1) Is it safe to do so (i.e. is Voldemort's client thread safe)? I read and assumed it was.
2) I am seeing small memory leaks on the server, while all my "server" threads are cleaned up nicely according to my profiler. Is there a known leak/issue here with the Voldemort clients in 0.90.1? Or should I give every thread its own client instance from a factory object (but how do I close them explicitly then)?
3) Of course the behavior differs across JDKs: OS X seems to have almost no problems, Linux with OpenJDK6 is the worst, Sun JDK6 on Linux also has the problem, but the leak is smaller per request. Just to make life easier.... But the leak exists. The difference between OS X and Linux is striking in the sense that I've also had to up ulimit for processes on Linux (not OS X) to a high value ( I chose 51200, just to get rid of out of file descriptors).
FWIW, I create the SocketStoreClientFactory with this client config (in Scala) - clueless why, but these settings seem to work:
1) AFAIK it's safe. At least I'm running a multithread server and I've not seen any problem.
2) As you already commented in another thread of the group, I've already found and fixed some memory and socked leaks. You can find the proposed patch in https://github.com/voldemort/voldemort/pull/70
3) I also needed to increase my ulimit to avoid "Too Many Open File" errors, but the previous patch seems to be solving the issues
I hope it helps.
Carlos
On Wed, Apr 4, 2012 at 4:57 PM, Maarten Koopmans <maarten.koopm...@gmail.com
> The list is a bit quiet - I hope somebody can shine light on this - > even if I'm totally off track here.
> I have a multi-threaded server that acts as a client to Voldemort, The > threads are more like actors/processes, in that they share nothing > except the Voldemort clients. These are obtained from a singleton, and > are shared across threads.
> 1) Is it safe to do so (i.e. is Voldemort's client thread safe)? I > read and assumed it was.
> 2) I am seeing small memory leaks on the server, while all my "server" > threads are cleaned up nicely according to my profiler. Is there a > known leak/issue here with the Voldemort clients in 0.90.1? Or should > I give every thread its own client instance from a factory object (but > how do I close them explicitly then)?
> 3) Of course the behavior differs across JDKs: OS X seems to have > almost no problems, Linux with OpenJDK6 is the worst, Sun JDK6 on > Linux also has the problem, but the leak is smaller per request. Just > to make life easier.... But the leak exists. The difference between OS > X and Linux is striking in the sense that I've also had to up ulimit > for processes on Linux (not OS X) to a high value ( I chose 51200, > just to get rid of out of file descriptors).
> FWIW, I create the SocketStoreClientFactory with this client config > (in Scala) - clueless why, but these settings seem to work:
> And a typical factory method for the singleton looks like this:
> var requestCounterClient: VMClient[String,Long] = null
> def getRequestMeterer() = { > if (requestCounterClient == null) { > debug("Getting VM request client") > val tmp: StoreClient[String,Long]= factory > getStoreClient("requests") > requestCounterClient = new > VMClient[String,Long](tmp) > } > debug("requestCounterClient - null?" + > (requestCounterClient == null))
> requestCounterClient > }
> VMClient is a scala wrapper around LazyStoreClient, and the > requestCounterClient is null until the first request comes in.
> If I should take a different approach or am missing something .... any > help/insights/advice greatly appreciated.
> Thanks,
> Maarten
> -- > You received this message because you are subscribed to the Google Groups > "project-voldemort" group. > To post to this group, send email to project-voldemort@googlegroups.com. > To unsubscribe from this group, send email to > project-voldemort+unsubscribe@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/project-voldemort?hl=en.
On Wednesday, April 4, 2012, Carlos Tasada wrote: > Hi Maarten,
> Answering your questions:
> 1) AFAIK it's safe. At least I'm running a multithread server and I've not > seen any problem.
> 2) As you already commented in another thread of the group, I've already > found and fixed some memory and socked leaks. You can find the proposed > patch in https://github.com/voldemort/voldemort/pull/70
> 3) I also needed to increase my ulimit to avoid "Too Many Open File" > errors, but the previous patch seems to be solving the issues
> I hope it helps.
> Carlos
> On Wed, Apr 4, 2012 at 4:57 PM, Maarten Koopmans < > maarten.koopm...@gmail.com <javascript:_e({}, 'cvml', > 'maarten.koopm...@gmail.com');>> wrote:
>> Hi,
>> The list is a bit quiet - I hope somebody can shine light on this - >> even if I'm totally off track here.
>> I have a multi-threaded server that acts as a client to Voldemort, The >> threads are more like actors/processes, in that they share nothing >> except the Voldemort clients. These are obtained from a singleton, and >> are shared across threads.
>> 1) Is it safe to do so (i.e. is Voldemort's client thread safe)? I >> read and assumed it was.
>> 2) I am seeing small memory leaks on the server, while all my "server" >> threads are cleaned up nicely according to my profiler. Is there a >> known leak/issue here with the Voldemort clients in 0.90.1? Or should >> I give every thread its own client instance from a factory object (but >> how do I close them explicitly then)?
>> 3) Of course the behavior differs across JDKs: OS X seems to have >> almost no problems, Linux with OpenJDK6 is the worst, Sun JDK6 on >> Linux also has the problem, but the leak is smaller per request. Just >> to make life easier.... But the leak exists. The difference between OS >> X and Linux is striking in the sense that I've also had to up ulimit >> for processes on Linux (not OS X) to a high value ( I chose 51200, >> just to get rid of out of file descriptors).
>> FWIW, I create the SocketStoreClientFactory with this client config >> (in Scala) - clueless why, but these settings seem to work:
>> And a typical factory method for the singleton looks like this:
>> var requestCounterClient: VMClient[String,Long] = null
>> def getRequestMeterer() = { >> if (requestCounterClient == null) { >> debug("Getting VM request client") >> val tmp: StoreClient[String,Long]= factory >> getStoreClient("requests") >> requestCounterClient = new >> VMClient[String,Long](tmp) >> } >> debug("requestCounterClient - null?" + >> (requestCounterClient == null))
>> requestCounterClient >> }
>> VMClient is a scala wrapper around LazyStoreClient, and the >> requestCounterClient is null until the first request comes in.
>> If I should take a different approach or am missing something .... any >> help/insights/advice greatly appreciated.
>> Thanks,
>> Maarten
>> -- >> You received this message because you are subscribed to the Google Groups >> "project-voldemort" group. >> To post to this group, send email to project-voldemort@googlegroups.com<javascript:_e({}, 'cvml', 'project-voldemort@googlegroups.com');> >> . >> To unsubscribe from this group, send email to >> project-voldemort+unsubscribe@googlegroups.com <javascript:_e({}, >> 'cvml', 'project-voldemort%2Bunsubscribe@googlegroups.com');>. >> For more options, visit this group at >> http://groups.google.com/group/project-voldemort?hl=en.
> -- > You received this message because you are subscribed to the Google Groups > "project-voldemort" group. > To post to this group, send email to project-voldemort@googlegroups.com<javascript:_e({}, 'cvml', 'project-voldemort@googlegroups.com');> > . > To unsubscribe from this group, send email to > project-voldemort+unsubscribe@googlegroups.com <javascript:_e({}, 'cvml', > 'project-voldemort%2Bunsubscribe@googlegroups.com');>. > For more options, visit this group at > http://groups.google.com/group/project-voldemort?hl=en.
Even when the patch is for the master branch, you can apply it to 0.90.1 easily. In fact I've it applied there in my production code.
AFAIK there are no other leaks in the connection code, but I known there're some points that could be improved (some of them are already modified in the master branch)
Regards, Carlos
On Wed, Apr 4, 2012 at 7:03 PM, Maarten Koopmans <maarten.koopm...@gmail.com
> Many thanks, I thought it was me. Did you apply the patch against 0.90.1 > or do I need to build against the current master branch?
> Thanks again, I'll report my findings tomorrow or early next week ( I am > in GMT+1).
> And I hope the core team will be a bit more responsive, if only to > incorporate patches and make releases....
> Thanks again,
> Maarten
> On Wednesday, April 4, 2012, Carlos Tasada wrote:
>> Hi Maarten,
>> Answering your questions:
>> 1) AFAIK it's safe. At least I'm running a multithread server and I've >> not seen any problem.
>> 2) As you already commented in another thread of the group, I've already >> found and fixed some memory and socked leaks. You can find the proposed >> patch in https://github.com/voldemort/voldemort/pull/70
>> 3) I also needed to increase my ulimit to avoid "Too Many Open File" >> errors, but the previous patch seems to be solving the issues
>> I hope it helps.
>> Carlos
>> On Wed, Apr 4, 2012 at 4:57 PM, Maarten Koopmans < >> maarten.koopm...@gmail.com> wrote:
>>> Hi,
>>> The list is a bit quiet - I hope somebody can shine light on this - >>> even if I'm totally off track here.
>>> I have a multi-threaded server that acts as a client to Voldemort, The >>> threads are more like actors/processes, in that they share nothing >>> except the Voldemort clients. These are obtained from a singleton, and >>> are shared across threads.
>>> 1) Is it safe to do so (i.e. is Voldemort's client thread safe)? I >>> read and assumed it was.
>>> 2) I am seeing small memory leaks on the server, while all my "server" >>> threads are cleaned up nicely according to my profiler. Is there a >>> known leak/issue here with the Voldemort clients in 0.90.1? Or should >>> I give every thread its own client instance from a factory object (but >>> how do I close them explicitly then)?
>>> 3) Of course the behavior differs across JDKs: OS X seems to have >>> almost no problems, Linux with OpenJDK6 is the worst, Sun JDK6 on >>> Linux also has the problem, but the leak is smaller per request. Just >>> to make life easier.... But the leak exists. The difference between OS >>> X and Linux is striking in the sense that I've also had to up ulimit >>> for processes on Linux (not OS X) to a high value ( I chose 51200, >>> just to get rid of out of file descriptors).
>>> FWIW, I create the SocketStoreClientFactory with this client config >>> (in Scala) - clueless why, but these settings seem to work:
>>> And a typical factory method for the singleton looks like this:
>>> var requestCounterClient: VMClient[String,Long] = null
>>> def getRequestMeterer() = { >>> if (requestCounterClient == null) { >>> debug("Getting VM request client") >>> val tmp: StoreClient[String,Long]= >>> factory getStoreClient("requests") >>> requestCounterClient = new >>> VMClient[String,Long](tmp) >>> } >>> debug("requestCounterClient - null?" + >>> (requestCounterClient == null))
>>> requestCounterClient >>> }
>>> VMClient is a scala wrapper around LazyStoreClient, and the >>> requestCounterClient is null until the first request comes in.
>>> If I should take a different approach or am missing something .... any >>> help/insights/advice greatly appreciated.
>>> Thanks,
>>> Maarten
>>> -- >>> You received this message because you are subscribed to the Google >>> Groups "project-voldemort" group. >>> To post to this group, send email to project-voldemort@googlegroups.com. >>> To unsubscribe from this group, send email to >>> project-voldemort+unsubscribe@googlegroups.com. >>> For more options, visit this group at >>> http://groups.google.com/group/project-voldemort?hl=en.
>> -- >> You received this message because you are subscribed to the Google Groups >> "project-voldemort" group. >> To post to this group, send email to project-voldemort@googlegroups.com. >> To unsubscribe from this group, send email to >> project-voldemort+unsubscribe@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/project-voldemort?hl=en.
> -- > You received this message because you are subscribed to the Google Groups > "project-voldemort" group. > To post to this group, send email to project-voldemort@googlegroups.com. > To unsubscribe from this group, send email to > project-voldemort+unsubscribe@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/project-voldemort?hl=en.
Late reply, but the patch worked. With a subtle memory leak caused by
the Scala compiler in my software all runs smoothly. Only thing that I
find slightly worrisome is that the community seems to slowly get to a
halt.
Sure Voldemort largely just works, but if nobody asks questions then
that suggest not many new users - the lack of roadmap for releases
etc. is also not helping. Or is this a quiet period in the US in terms
of holidays or so?
On Wed, Apr 4, 2012 at 7:41 PM, Carlos Tasada <ctas...@gmail.com> wrote:
> Hi Marteen,
> Even when the patch is for the master branch, you can apply it to 0.90.1
> easily. In fact I've it applied there in my production code.
> AFAIK there are no other leaks in the connection code, but I known there're
> some points that could be improved (some of them are already modified in the
> master branch)
> Regards,
> Carlos
> On Wed, Apr 4, 2012 at 7:03 PM, Maarten Koopmans
> <maarten.koopm...@gmail.com> wrote:
>> Hi Carlos,
>> Many thanks, I thought it was me. Did you apply the patch against 0.90.1
>> or do I need to build against the current master branch?
>> Thanks again, I'll report my findings tomorrow or early next week ( I am
>> in GMT+1).
>> And I hope the core team will be a bit more responsive, if only to
>> incorporate patches and make releases....
>> Thanks again,
>> Maarten
>> On Wednesday, April 4, 2012, Carlos Tasada wrote:
>>> Hi Maarten,
>>> Answering your questions:
>>> 1) AFAIK it's safe. At least I'm running a multithread server and I've
>>> not seen any problem.
>>> 2) As you already commented in another thread of the group, I've already
>>> found and fixed some memory and socked leaks. You can find the proposed
>>> patch in https://github.com/voldemort/voldemort/pull/70
>>> 3) I also needed to increase my ulimit to avoid "Too Many Open File"
>>> errors, but the previous patch seems to be solving the issues
>>> I hope it helps.
>>> Carlos
>>> On Wed, Apr 4, 2012 at 4:57 PM, Maarten Koopmans
>>> <maarten.koopm...@gmail.com> wrote:
>>>> Hi,
>>>> The list is a bit quiet - I hope somebody can shine light on this -
>>>> even if I'm totally off track here.
>>>> I have a multi-threaded server that acts as a client to Voldemort, The
>>>> threads are more like actors/processes, in that they share nothing
>>>> except the Voldemort clients. These are obtained from a singleton, and
>>>> are shared across threads.
>>>> 1) Is it safe to do so (i.e. is Voldemort's client thread safe)? I
>>>> read and assumed it was.
>>>> 2) I am seeing small memory leaks on the server, while all my "server"
>>>> threads are cleaned up nicely according to my profiler. Is there a
>>>> known leak/issue here with the Voldemort clients in 0.90.1? Or should
>>>> I give every thread its own client instance from a factory object (but
>>>> how do I close them explicitly then)?
>>>> 3) Of course the behavior differs across JDKs: OS X seems to have
>>>> almost no problems, Linux with OpenJDK6 is the worst, Sun JDK6 on
>>>> Linux also has the problem, but the leak is smaller per request. Just
>>>> to make life easier.... But the leak exists. The difference between OS
>>>> X and Linux is striking in the sense that I've also had to up ulimit
>>>> for processes on Linux (not OS X) to a high value ( I chose 51200,
>>>> just to get rid of out of file descriptors).
>>>> FWIW, I create the SocketStoreClientFactory with this client config
>>>> (in Scala) - clueless why, but these settings seem to work:
>>>> And a typical factory method for the singleton looks like this:
>>>> var requestCounterClient: VMClient[String,Long] = null
>>>> def getRequestMeterer() = {
>>>> if (requestCounterClient == null) {
>>>> debug("Getting VM request client")
>>>> val tmp: StoreClient[String,Long]=
>>>> factory getStoreClient("requests")
>>>> requestCounterClient = new
>>>> VMClient[String,Long](tmp)
>>>> }
>>>> debug("requestCounterClient - null?" +
>>>> (requestCounterClient == null))
>>>> requestCounterClient
>>>> }
>>>> VMClient is a scala wrapper around LazyStoreClient, and the
>>>> requestCounterClient is null until the first request comes in.
>>>> If I should take a different approach or am missing something .... any
>>>> help/insights/advice greatly appreciated.
>>>> Thanks,
>>>> Maarten
>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "project-voldemort" group.
>>>> To post to this group, send email to project-voldemort@googlegroups.com.
>>>> To unsubscribe from this group, send email to
>>>> project-voldemort+unsubscribe@googlegroups.com.
>>>> For more options, visit this group at
>>>> http://groups.google.com/group/project-voldemort?hl=en.
>>> --
>>> You received this message because you are subscribed to the Google Groups
>>> "project-voldemort" group.
>>> To post to this group, send email to project-voldemort@googlegroups.com.
>>> To unsubscribe from this group, send email to
>>> project-voldemort+unsubscribe@googlegroups.com.
>>> For more options, visit this group at
>>> http://groups.google.com/group/project-voldemort?hl=en.
>> --
>> You received this message because you are subscribed to the Google Groups
>> "project-voldemort" group.
>> To post to this group, send email to project-voldemort@googlegroups.com.
>> To unsubscribe from this group, send email to
>> project-voldemort+unsubscribe@googlegroups.com.
>> For more options, visit this group at
>> http://groups.google.com/group/project-voldemort?hl=en.
> --
> You received this message because you are subscribed to the Google Groups
> "project-voldemort" group.
> To post to this group, send email to project-voldemort@googlegroups.com.
> To unsubscribe from this group, send email to
> project-voldemort+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/project-voldemort?hl=en.
I'm happy to heard the patch worked. Those fixes will be available in the
next Voldemort version.
As you say the newsgroup is really quiet, but if you check in the project
github page, there is some activity there. Seems that version 0.95 is
already in the oven ;)
And regarding the roadmap, I think it's also a problem related with the
maintenance of the opened issues. The impression is that the project is not
really transparent with the community, or at least that's my impression. A
different thing is the feedback from the developers that so far has been
great, at least in my personal experience.
Regards,
Carlos.
On Sat, Apr 28, 2012 at 10:11 AM, Maarten Koopmans <
> Late reply, but the patch worked. With a subtle memory leak caused by
> the Scala compiler in my software all runs smoothly. Only thing that I
> find slightly worrisome is that the community seems to slowly get to a
> halt.
> Sure Voldemort largely just works, but if nobody asks questions then
> that suggest not many new users - the lack of roadmap for releases
> etc. is also not helping. Or is this a quiet period in the US in terms
> of holidays or so?
> Best regards, Maarten
> On Wed, Apr 4, 2012 at 7:41 PM, Carlos Tasada <ctas...@gmail.com> wrote:
> > Hi Marteen,
> > Even when the patch is for the master branch, you can apply it to 0.90.1
> > easily. In fact I've it applied there in my production code.
> > AFAIK there are no other leaks in the connection code, but I known
> there're
> > some points that could be improved (some of them are already modified in
> the
> > master branch)
> > Regards,
> > Carlos
> > On Wed, Apr 4, 2012 at 7:03 PM, Maarten Koopmans
> > <maarten.koopm...@gmail.com> wrote:
> >> Hi Carlos,
> >> Many thanks, I thought it was me. Did you apply the patch against 0.90.1
> >> or do I need to build against the current master branch?
> >> Thanks again, I'll report my findings tomorrow or early next week ( I am
> >> in GMT+1).
> >> And I hope the core team will be a bit more responsive, if only to
> >> incorporate patches and make releases....
> >> Thanks again,
> >> Maarten
> >> On Wednesday, April 4, 2012, Carlos Tasada wrote:
> >>> Hi Maarten,
> >>> Answering your questions:
> >>> 1) AFAIK it's safe. At least I'm running a multithread server and I've
> >>> not seen any problem.
> >>> 2) As you already commented in another thread of the group, I've
> already
> >>> found and fixed some memory and socked leaks. You can find the proposed
> >>> patch in https://github.com/voldemort/voldemort/pull/70
> >>> 3) I also needed to increase my ulimit to avoid "Too Many Open File"
> >>> errors, but the previous patch seems to be solving the issues
> >>> I hope it helps.
> >>> Carlos
> >>> On Wed, Apr 4, 2012 at 4:57 PM, Maarten Koopmans
> >>> <maarten.koopm...@gmail.com> wrote:
> >>>> Hi,
> >>>> The list is a bit quiet - I hope somebody can shine light on this -
> >>>> even if I'm totally off track here.
> >>>> I have a multi-threaded server that acts as a client to Voldemort, The
> >>>> threads are more like actors/processes, in that they share nothing
> >>>> except the Voldemort clients. These are obtained from a singleton, and
> >>>> are shared across threads.
> >>>> 1) Is it safe to do so (i.e. is Voldemort's client thread safe)? I
> >>>> read and assumed it was.
> >>>> 2) I am seeing small memory leaks on the server, while all my "server"
> >>>> threads are cleaned up nicely according to my profiler. Is there a
> >>>> known leak/issue here with the Voldemort clients in 0.90.1? Or should
> >>>> I give every thread its own client instance from a factory object (but
> >>>> how do I close them explicitly then)?
> >>>> 3) Of course the behavior differs across JDKs: OS X seems to have
> >>>> almost no problems, Linux with OpenJDK6 is the worst, Sun JDK6 on
> >>>> Linux also has the problem, but the leak is smaller per request. Just
> >>>> to make life easier.... But the leak exists. The difference between OS
> >>>> X and Linux is striking in the sense that I've also had to up ulimit
> >>>> for processes on Linux (not OS X) to a high value ( I chose 51200,
> >>>> just to get rid of out of file descriptors).
> >>>> FWIW, I create the SocketStoreClientFactory with this client config
> >>>> (in Scala) - clueless why, but these settings seem to work:
> >>>> VMClient is a scala wrapper around LazyStoreClient, and the
> >>>> requestCounterClient is null until the first request comes in.
> >>>> If I should take a different approach or am missing something .... any
> >>>> help/insights/advice greatly appreciated.
> >>>> Thanks,
> >>>> Maarten
> >>>> --
> >>>> You received this message because you are subscribed to the Google
> >>>> Groups "project-voldemort" group.
> >>>> To post to this group, send email to
> project-voldemort@googlegroups.com.
> >>>> To unsubscribe from this group, send email to
> >>>> project-voldemort+unsubscribe@googlegroups.com.
> >>>> For more options, visit this group at
> >>>> http://groups.google.com/group/project-voldemort?hl=en.
> >>> --
> >>> You received this message because you are subscribed to the Google
> Groups
> >>> "project-voldemort" group.
> >>> To post to this group, send email to
> project-voldemort@googlegroups.com.
> >>> To unsubscribe from this group, send email to
> >>> project-voldemort+unsubscribe@googlegroups.com.
> >>> For more options, visit this group at
> >>> http://groups.google.com/group/project-voldemort?hl=en.
> >> --
> >> You received this message because you are subscribed to the Google
> Groups
> >> "project-voldemort" group.
> >> To post to this group, send email to project-voldemort@googlegroups.com
> .
> >> To unsubscribe from this group, send email to
> >> project-voldemort+unsubscribe@googlegroups.com.
> >> For more options, visit this group at
> >> http://groups.google.com/group/project-voldemort?hl=en.
> > --
> > You received this message because you are subscribed to the Google Groups
> > "project-voldemort" group.
> > To post to this group, send email to project-voldemort@googlegroups.com.
> > To unsubscribe from this group, send email to
> > project-voldemort+unsubscribe@googlegroups.com.
> > For more options, visit this group at
> > http://groups.google.com/group/project-voldemort?hl=en.
> --
> You received this message because you are subscribed to the Google Groups
> "project-voldemort" group.
> To post to this group, send email to project-voldemort@googlegroups.com.
> To unsubscribe from this group, send email to
> project-voldemort+unsubscribe@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/project-voldemort?hl=en.
Thanks Carlos for helping out with the patch and for the knowledge
sharing.
@ Maarten : I can understand the general frustration regarding late
replies in the Voldemort community. There would be a release out soon
including the patch submitted by Carlos. And yes, we will also try to
better communicate with respect to releases.
The reality is that there are a lot of open issues in the LinkedIn
world keeping the few devs busy. FYI: we're now aggressively looking
into the pending pull requests on github (since that is of higher
priority). In time, we hope to catch up with the Google group
questions as well.
On Apr 28, 4:38 am, Carlos Tasada <ctas...@gmail.com> wrote:
> I'm happy to heard the patch worked. Those fixes will be available in the
> next Voldemort version.
> As you say the newsgroup is really quiet, but if you check in the project
> github page, there is some activity there. Seems that version 0.95 is
> already in the oven ;)
> And regarding the roadmap, I think it's also a problem related with the
> maintenance of the opened issues. The impression is that the project is not
> really transparent with the community, or at least that's my impression. A
> different thing is the feedback from the developers that so far has been
> great, at least in my personal experience.
> Regards,
> Carlos.
> On Sat, Apr 28, 2012 at 10:11 AM, Maarten Koopmans <
> > Late reply, but the patch worked. With a subtle memory leak caused by
> > the Scala compiler in my software all runs smoothly. Only thing that I
> > find slightly worrisome is that the community seems to slowly get to a
> > halt.
> > Sure Voldemort largely just works, but if nobody asks questions then
> > that suggest not many new users - the lack of roadmap for releases
> > etc. is also not helping. Or is this a quiet period in the US in terms
> > of holidays or so?
> > Best regards, Maarten
> > On Wed, Apr 4, 2012 at 7:41 PM, Carlos Tasada <ctas...@gmail.com> wrote:
> > > Hi Marteen,
> > > Even when the patch is for the master branch, you can apply it to 0.90.1
> > > easily. In fact I've it applied there in my production code.
> > > AFAIK there are no other leaks in the connection code, but I known
> > there're
> > > some points that could be improved (some of them are already modified in
> > the
> > > master branch)
> > > Regards,
> > > Carlos
> > > On Wed, Apr 4, 2012 at 7:03 PM, Maarten Koopmans
> > > <maarten.koopm...@gmail.com> wrote:
> > >> Hi Carlos,
> > >> Many thanks, I thought it was me. Did you apply the patch against 0.90.1
> > >> or do I need to build against the current master branch?
> > >> Thanks again, I'll report my findings tomorrow or early next week ( I am
> > >> in GMT+1).
> > >> And I hope the core team will be a bit more responsive, if only to
> > >> incorporate patches and make releases....
> > >> Thanks again,
> > >> Maarten
> > >> On Wednesday, April 4, 2012, Carlos Tasada wrote:
> > >>> Hi Maarten,
> > >>> Answering your questions:
> > >>> 1) AFAIK it's safe. At least I'm running a multithread server and I've
> > >>> not seen any problem.
> > >>> 2) As you already commented in another thread of the group, I've
> > already
> > >>> found and fixed some memory and socked leaks. You can find the proposed
> > >>> patch in https://github.com/voldemort/voldemort/pull/70
> > >>> 3) I also needed to increase my ulimit to avoid "Too Many Open File"
> > >>> errors, but the previous patch seems to be solving the issues
> > >>> I hope it helps.
> > >>> Carlos
> > >>> On Wed, Apr 4, 2012 at 4:57 PM, Maarten Koopmans
> > >>> <maarten.koopm...@gmail.com> wrote:
> > >>>> Hi,
> > >>>> The list is a bit quiet - I hope somebody can shine light on this -
> > >>>> even if I'm totally off track here.
> > >>>> I have a multi-threaded server that acts as a client to Voldemort, The
> > >>>> threads are more like actors/processes, in that they share nothing
> > >>>> except the Voldemort clients. These are obtained from a singleton, and
> > >>>> are shared across threads.
> > >>>> 1) Is it safe to do so (i.e. is Voldemort's client thread safe)? I
> > >>>> read and assumed it was.
> > >>>> 2) I am seeing small memory leaks on the server, while all my "server"
> > >>>> threads are cleaned up nicely according to my profiler. Is there a
> > >>>> known leak/issue here with the Voldemort clients in 0.90.1? Or should
> > >>>> I give every thread its own client instance from a factory object (but
> > >>>> how do I close them explicitly then)?
> > >>>> 3) Of course the behavior differs across JDKs: OS X seems to have
> > >>>> almost no problems, Linux with OpenJDK6 is the worst, Sun JDK6 on
> > >>>> Linux also has the problem, but the leak is smaller per request. Just
> > >>>> to make life easier.... But the leak exists. The difference between OS
> > >>>> X and Linux is striking in the sense that I've also had to up ulimit
> > >>>> for processes on Linux (not OS X) to a high value ( I chose 51200,
> > >>>> just to get rid of out of file descriptors).
> > >>>> FWIW, I create the SocketStoreClientFactory with this client config
> > >>>> (in Scala) - clueless why, but these settings seem to work:
> > >>>> VMClient is a scala wrapper around LazyStoreClient, and the
> > >>>> requestCounterClient is null until the first request comes in.
> > >>>> If I should take a different approach or am missing something .... any
> > >>>> help/insights/advice greatly appreciated.
> > >>>> Thanks,
> > >>>> Maarten
> > >>>> --
> > >>>> You received this message because you are subscribed to the Google
> > >>>> Groups "project-voldemort" group.
> > >>>> To post to this group, send email to
> > project-voldemort@googlegroups.com.
> > >>>> To unsubscribe from this group, send email to
> > >>>> project-voldemort+unsubscribe@googlegroups.com.
> > >>>> For more options, visit this group at
> > >>>>http://groups.google.com/group/project-voldemort?hl=en.
> > >>> --
> > >>> You received this message because you are subscribed to the Google
> > Groups
> > >>> "project-voldemort" group.
> > >>> To post to this group, send email to
> > project-voldemort@googlegroups.com.
> > >>> To unsubscribe from this group, send email to
> > >>> project-voldemort+unsubscribe@googlegroups.com.
> > >>> For more options, visit this group at
> > >>>http://groups.google.com/group/project-voldemort?hl=en.
> > >> --
> > >> You received this message because you are subscribed to the Google
> > Groups
> > >> "project-voldemort" group.
> > >> To post to this group, send email to project-voldemort@googlegroups.com
> > .
> > >> To unsubscribe from this group, send email to
> > >> project-voldemort+unsubscribe@googlegroups.com.
> > >> For more options, visit this group at
> > >>http://groups.google.com/group/project-voldemort?hl=en.
> > > --
> > > You received this message because you are subscribed to the Google Groups
> > > "project-voldemort" group.
> > > To post to this group, send email to project-voldemort@googlegroups.com.
> > > To unsubscribe from this group, send email to
> > > project-voldemort+unsubscribe@googlegroups.com.
> > > For more options, visit this group at
> > >http://groups.google.com/group/project-voldemort?hl=en.
> > --
> > You received this message because you are subscribed to the Google Groups
> > "project-voldemort" group.
> > To post to this group, send email to project-voldemort@googlegroups.com.
> > To unsubscribe from this group, send email to
> > project-voldemort+unsubscribe@googlegroups.com.
> > For more options, visit this group at
> >http://groups.google.com/group/project-voldemort?hl=en.
Good to read, catching up.... Voldemort is such an exceptional tedhnology,
that it would be painful to watch it go out. I'm happy that's not happening!
On Monday, April 30, 2012, Chinmay Soman wrote:
> Thanks Carlos for helping out with the patch and for the knowledge
> sharing.
> @ Maarten : I can understand the general frustration regarding late
> replies in the Voldemort community. There would be a release out soon
> including the patch submitted by Carlos. And yes, we will also try to
> better communicate with respect to releases.
> The reality is that there are a lot of open issues in the LinkedIn
> world keeping the few devs busy. FYI: we're now aggressively looking
> into the pending pull requests on github (since that is of higher
> priority). In time, we hope to catch up with the Google group
> questions as well.
> On Apr 28, 4:38 am, Carlos Tasada <ctas...@gmail.com <javascript:;>>
> wrote:
> > Hi Maarten,
> > I'm happy to heard the patch worked. Those fixes will be available in the
> > next Voldemort version.
> > As you say the newsgroup is really quiet, but if you check in the project
> > github page, there is some activity there. Seems that version 0.95 is
> > already in the oven ;)
> > And regarding the roadmap, I think it's also a problem related with the
> > maintenance of the opened issues. The impression is that the project is
> not
> > really transparent with the community, or at least that's my impression.
> A
> > different thing is the feedback from the developers that so far has been
> > great, at least in my personal experience.
> > Regards,
> > Carlos.
> > On Sat, Apr 28, 2012 at 10:11 AM, Maarten Koopmans <
> > > Late reply, but the patch worked. With a subtle memory leak caused by
> > > the Scala compiler in my software all runs smoothly. Only thing that I
> > > find slightly worrisome is that the community seems to slowly get to a
> > > halt.
> > > Sure Voldemort largely just works, but if nobody asks questions then
> > > that suggest not many new users - the lack of roadmap for releases
> > > etc. is also not helping. Or is this a quiet period in the US in terms
> > > of holidays or so?
> > > Best regards, Maarten
> > > On Wed, Apr 4, 2012 at 7:41 PM, Carlos Tasada <ctas...@gmail.com<javascript:;>>
> wrote:
> > > > Hi Marteen,
> > > > Even when the patch is for the master branch, you can apply it to
> 0.90.1
> > > > easily. In fact I've it applied there in my production code.
> > > > AFAIK there are no other leaks in the connection code, but I known
> > > there're
> > > > some points that could be improved (some of them are already
> modified in
> > > the
> > > > master branch)
> > > >> Many thanks, I thought it was me. Did you apply the patch against
> 0.90.1
> > > >> or do I need to build against the current master branch?
> > > >> Thanks again, I'll report my findings tomorrow or early next week (
> I am
> > > >> in GMT+1).
> > > >> And I hope the core team will be a bit more responsive, if only to
> > > >> incorporate patches and make releases....
> > > >> Thanks again,
> > > >> Maarten
> > > >> On Wednesday, April 4, 2012, Carlos Tasada wrote:
> > > >>> Hi Maarten,
> > > >>> Answering your questions:
> > > >>> 1) AFAIK it's safe. At least I'm running a multithread server and
> I've
> > > >>> not seen any problem.
> > > >>> 2) As you already commented in another thread of the group, I've
> > > already
> > > >>> found and fixed some memory and socked leaks. You can find the
> proposed
> > > >>> patch in https://github.com/voldemort/voldemort/pull/70
> > > >>> 3) I also needed to increase my ulimit to avoid "Too Many Open
> File"
> > > >>> errors, but the previous patch seems to be solving the issues
> > > >>> I hope it helps.
> > > >>> Carlos
> > > >>> On Wed, Apr 4, 2012 at 4:57 PM, Maarten Koopmans
> > > >>> <maarten.koopm...@gmail.com> wrote:
> > > >>>> Hi,
> > > >>>> The list is a bit quiet - I hope somebody can shine light on this
> -
> > > >>>> even if I'm totally off track here.
> > > >>>> I have a multi-threaded server that acts as a client to
> Voldemort, The
> > > >>>> threads are more like actors/processes, in that they share nothing
> > > >>>> except the Voldemort clients. These are obtained from a
> singleton, and
> > > >>>> are shared across threads.
> > > >>>> 1) Is it safe to do so (i.e. is Voldemort's client thread safe)? I
> > > >>>> read and assumed it was.
> > > >>>> 2) I am seeing small memory leaks on the server, while all my
> "server"
> > > >>>> threads are cleaned up nicely according to my profiler. Is there a
> > > >>>> known leak/issue here with the Voldemort clients in 0.90.1? Or
> should
> > > >>>> I give every thread its own client instance from a factory object
> (but
> > > >>>> how do I close them explicitly then)?
> > > >>>> 3) Of course the behavior differs across JDKs: OS X seems to have
> > > >>>> almost no problems, Linux with OpenJDK6 is the worst, Sun JDK6 on
> > > >>>> Linux also has the problem, but the leak is smaller per request.
> Just
> > > >>>> to make life easier.... But the leak exists. The difference
> between OS
> > > >>>> X and Linux is striking in the sense that I've also had to up
> ulimit
> > > >>>> for processes on Linux (not OS X) to a high value ( I chose 51200,
> > > >>>> just to get rid of out of file descriptors).
> > > >>>> FWIW, I create the SocketStoreClientFactory with this client
> config
> > > >>>> (in Scala) - clueless why, but these settings seem to work:
