I saw that in the recent months most of the development was going
towards replacing libev with libevent. Probably there is a reason for
this and I, and I guess the other gevent fans, will be happy to learn
it. Is libev faster, simpler to use, has http server that supports
streaming, etc?
I might be totally wrong, but the thing that worries me the most, is that switching to libev will make it impossible to get decent performance from gevent on Windows. I personally don't care much about that platform, but my customer has decided to use it for their applications and I have to support them.
Libevent on Windows, uses I/O Completion Ports (IOCP) and scales beautifully. From the link above, it looks like libev is using the old select call and it is well known that this will not scale beyond a few hundred sockets, on either Windows or Linux.
It looks like switching to libev will make gevent a UNIX-library only, but I might be wrong.
On Tue, Apr 26, 2011 at 17:43, galfy <galfyo.pun...@googlemail.com> wrote: > I saw that in the recent months most of the development was going > towards replacing libev with libevent. Probably there is a reason for > this and I, and I guess the other gevent fans, will be happy to learn > it. Is libev faster, simpler to use, has http server that supports > streaming, etc?
-- Jan Persson - Esentus Technology AB - www.esentus.com - +46 702 854132 (mobile)
On Apr 26, 2:09 pm, Jan Persson <jan.pers...@gmail.com> wrote:
> I might be totally wrong, but the thing that worries me the most, is
> that switching to libev will make it impossible to get decent
> performance from gevent on Windows. I personally don't care much about
> that platform, but my customer has decided to use it for their
> applications and I have to support them.
> Libevent on Windows, uses I/O Completion Ports (IOCP) and scales
> beautifully. From the link above, it looks like libev is using the old
> select call and it is well known that this will not scale beyond a few
> hundred sockets, on either Windows or Linux.
> It looks like switching to libev will make gevent a UNIX-library only,
> but I might be wrong.
AFAICT you're basically correct. In order to support windows and unix
well, ry (node.js author) is working on https://github.com/joyent/liboio which he announced at CodeConf.
Once it matures it's probably not unreasonable for gevent to target it
and get the ability to support both windows and file i/o well.
On Wed, Apr 27, 2011 at 09:32, Travis Cline <travis.cl...@gmail.com> wrote: > On Apr 26, 2:09 pm, Jan Persson <jan.pers...@gmail.com> wrote: > AFAICT you're basically correct. In order to support windows and unix > well, ry (node.js author) is working on https://github.com/joyent/liboio > which he announced at CodeConf.
> Once it matures it's probably not unreasonable for gevent to target it > and get the ability to support both windows and file i/o well.
Thanks for the explanation. So the plan is basically to first make a transition to libev and then to liboio? I hope that those of us who still needs to support Windows will have the possible to change backend, in spite of this posting:
"Only libev will be supported in 0.14. However, other than gevent.core the interface of gevent package is not changed, so transition should be smooth."
I (and the original poster) still wonders why there is a switch to libev in the first place, when libevent is working? I rest assured that there are good reasons for this, but could Denis care to explain?
-- Jan Persson - Esentus Technology AB - www.esentus.com - +46 702 854132 (mobile)
> On Wed, Apr 27, 2011 at 09:32, Travis Cline <travis.cl...@gmail.com> wrote:
> > On Apr 26, 2:09 pm, Jan Persson <jan.pers...@gmail.com> wrote:
> > AFAICT you're basically correct. In order to support windows and unix
> > well, ry (node.js author) is working onhttps://github.com/joyent/liboio > > which he announced at CodeConf.
> > Once it matures it's probably not unreasonable for gevent to target it
> > and get the ability to support both windows and file i/o well.
> Thanks for the explanation. So the plan is basically to first make a
> transition to libev and then to liboio?
> I hope that those of us who still needs to support Windows will have
> the possible to change backend, in spite of this posting:
> "Only libev will be supported in 0.14. However, other than gevent.core
> the interface of gevent package is not changed, so transition should be smooth."
> I (and the original poster) still wonders why there is a switch to
> libev in the first place, when libevent is working? I rest assured
> that there are good reasons for this, but could Denis care to explain?
> --
> Jan Persson - Esentus Technology AB -www.esentus.com- +46 702 854132 (mobile)
On Thu, Apr 28, 2011 at 6:50 AM, Equand <equ...@gmail.com> wrote: > seems like problem with DNS arises with libevent only (their DNS > resolving is not fixed yet even in 2.0.)
> On Apr 27, 12:20 pm, Jan Persson <jan.pers...@gmail.com> wrote: >> On Wed, Apr 27, 2011 at 09:32, Travis Cline <travis.cl...@gmail.com> wrote: >> > On Apr 26, 2:09 pm, Jan Persson <jan.pers...@gmail.com> wrote: >> > AFAICT you're basically correct. In order to support windows and unix >> > well, ry (node.js author) is working onhttps://github.com/joyent/liboio >> > which he announced at CodeConf.
>> > Once it matures it's probably not unreasonable for gevent to target it >> > and get the ability to support both windows and file i/o well.
>> Thanks for the explanation. So the plan is basically to first make a >> transition to libev and then to liboio? >> I hope that those of us who still needs to support Windows will have >> the possible to change backend, in spite of this posting:
>> "Only libev will be supported in 0.14. However, other than gevent.core >> the interface of gevent package is not changed, so transition should be smooth."
>> I (and the original poster) still wonders why there is a switch to >> libev in the first place, when libevent is working? I rest assured >> that there are good reasons for this, but could Denis care to explain?
>> -- >> Jan Persson - Esentus Technology AB -www.esentus.com- +46 702 854132 (mobile)
On Tue, Apr 26, 2011 at 10:43 PM, galfy <galfyo.pun...@googlemail.com> wrote: > I saw that in the recent months most of the development was going > towards replacing libev with libevent. Probably there is a reason for > this and I, and I guess the other gevent fans, will be happy to learn > it. Is libev faster, simpler to use, has http server that supports > streaming, etc?
faster - probably somewhat, but that wasn't the reason for switch. simpler to use - yes, the API is really nicer. has http server - no, although a few servers have been written by 3rd parties.
On Wed, Apr 27, 2011 at 2:09 AM, Jan Persson <jan.pers...@gmail.com> wrote: > Libevent on Windows, uses I/O Completion Ports (IOCP) and scales > beautifully. From the link above, it looks like libev is using the old > select call and it is well known that this will not scale beyond a few > hundred sockets, on either Windows or Linux.
Libevent2 has support for IOCP, but it's not automatic, you need to use asynchronous bufferevents to use it. Gevent does not wrap bufferevents, let alone asynchronous. The libevent itself does not use async bufferevents in DNS, HTTP client and HTTP server.
Thus there's no way to take advantage of IOCP in gevent 0.13 and so switching to libev does not imply degraded performance on Windows (it remains poor).
> It looks like switching to libev will make gevent a UNIX-library only, > but I might be wrong.
It won't become UNIX-only, I think gevent 0.14 will support Windows at least to the same extent as gevent 0.13 supported it.
> On Wed, Apr 27, 2011 at 09:32, Travis Cline <travis.cl...@gmail.com> wrote: > ry (node.js author) is working on https://github.com/joyent/liboio > which he announced at CodeConf.
> Once it matures it's probably not unreasonable for gevent to target it > and get the ability to support both windows and file i/o well.
It wasn't the plan to use liboio as I started using libev before that was announced, but that's an interesting project to watch and when it matures we can consider using as a base layer for gevent.
> Thus there's no way to take advantage of IOCP in gevent 0.13 and so
> switching to libev does not imply degraded performance on Windows (it
> remains poor).
I've only had a chance so far to use gevent under Windows XP. I have
my DNS settings pointing to a localhost instance of BIND9 and I've
noticed that about 10% of the HTTP GET requests I send (using the non-
monkey patched httplib) fail with DNS errors. The majority of the
failures are either "unknown" or "request timed out." Is this a
symptom of the poor performance under Windows or do you believe my
problem lies elsewhere (perhaps with my BIND configuration -- which is
default in all ways)?
On Fri, Apr 29, 2011 at 12:45 AM, James Simmons <ja...@avatrion.com> wrote: >> Thus there's no way to take advantage of IOCP in gevent 0.13 and so >> switching to libev does not imply degraded performance on Windows (it >> remains poor).
> I've only had a chance so far to use gevent under Windows XP. I have > my DNS settings pointing to a localhost instance of BIND9 and I've > noticed that about 10% of the HTTP GET requests I send (using the non- > monkey patched httplib) fail with DNS errors. The majority of the > failures are either "unknown" or "request timed out." Is this a > symptom of the poor performance under Windows or do you believe my > problem lies elsewhere (perhaps with my BIND configuration -- which is > default in all ways)?
You are talking about gevent 0.13, right? I think this the kind of problem that is fixed in the trunk by switching to c-ares for DNS. However you have to wait while trunk's Windows bugs fixed before you can try that (or help fixing them).
On Apr 28, 10:58 am, Denis Bilenko <denis.bile...@gmail.com> wrote:
> You are talking about gevent 0.13, right?
> I think this the kind of problem that is fixed in the trunk by
> switching to c-ares for DNS.
> However you have to wait while trunk's Windows bugs fixed before you
> can try that (or help fixing them).
Yes I'm using 0.13. Glad to hear that the issues I'm experiencing are
fixed in trunk. I probably won't be of much use in fixing the issues
though unless it's something specific to the Python portion of the
project. I haven't had any luck getting it to compile (though I never
have any luck getting things to compile in Windows).
> Thus there's no way to take advantage of IOCP in gevent 0.13 and so > switching to libev does not imply degraded performance on Windows (it > remains poor).
Thanks for the answer. I was under the impression that IOCP was being used, since gevent has scaled so good for our Windows project (which has hundreds of clients) and this discussion made me a bit worried that the good days were over.
On Thu, Apr 28, 2011 at 7:32 PM, Denis Bilenko <denis.bile...@gmail.com> wrote: > On Tue, Apr 26, 2011 at 10:43 PM, galfy <galfyo.pun...@googlemail.com> wrote: >> I saw that in the recent months most of the development was going >> towards replacing libev with libevent. Probably there is a reason for >> this and I, and I guess the other gevent fans, will be happy to learn >> it. Is libev faster, simpler to use, has http server that supports >> streaming, etc?
> faster - probably somewhat, but that wasn't the reason for switch. > simpler to use - yes, the API is really nicer. > has http server - no, although a few servers have been written by 3rd parties.
> On Wed, Apr 27, 2011 at 2:09 AM, Jan Persson <jan.pers...@gmail.com> wrote: >> Libevent on Windows, uses I/O Completion Ports (IOCP) and scales >> beautifully. From the link above, it looks like libev is using the old >> select call and it is well known that this will not scale beyond a few >> hundred sockets, on either Windows or Linux.
> Libevent2 has support for IOCP, but it's not automatic, you need to > use asynchronous bufferevents to use it. > Gevent does not wrap bufferevents, let alone asynchronous. The > libevent itself does not use async bufferevents in DNS, HTTP client > and HTTP server.
> Thus there's no way to take advantage of IOCP in gevent 0.13 and so > switching to libev does not imply degraded performance on Windows (it > remains poor).
>> It looks like switching to libev will make gevent a UNIX-library only, >> but I might be wrong.
> It won't become UNIX-only, I think gevent 0.14 will support Windows at > least to the same extent as gevent 0.13 supported it.
>> On Wed, Apr 27, 2011 at 09:32, Travis Cline <travis.cl...@gmail.com> wrote: >> ry (node.js author) is working on https://github.com/joyent/liboio >> which he announced at CodeConf.
>> Once it matures it's probably not unreasonable for gevent to target it >> and get the ability to support both windows and file i/o well.
> It wasn't the plan to use liboio as I started using libev before that > was announced, but that's an interesting project to watch and when it > matures we can consider using as a base layer for gevent.
On Apr 28, 1:32 pm, Denis Bilenko <denis.bile...@gmail.com> wrote:
> faster - probably somewhat, but that wasn't the reason for switch.
> simpler to use - yes, the API is really nicer.
> has http server - no, although a few servers have been written by 3rd parties.
I'm also very interested in the impact of libev regarding wsgi support
of gevent. We've used libevent2 wsgi module instead of pure wsgi in
order to maximize the overall performance (which is really
noticeable).
Denis, do you have any plan to support fast, native wsgi when porting
to libev? Perhaps using third-party projects like https://github.com/jonashaag/bjoern which is also based on libev?
On 30 abr, 17:42, Andy <selforgani...@gmail.com> wrote:
> On Apr 28, 1:32 pm, Denis Bilenko <denis.bile...@gmail.com> wrote:
> > faster - probably somewhat, but that wasn't the reason for switch.
> > simpler to use - yes, the API is really nicer.
> > has http server - no, although a few servers have been written by 3rd parties.
> What http server will gevent 0.14 be using then?
On Sat, May 14, 2011 at 5:27 PM, Jaime <jjja...@gmail.com> wrote: > Denis, do you have any plan to support fast, native wsgi when porting > to libev? Perhaps using third-party projects like https://github.com/jonashaag/bjoern > which is also based on libev?
Yes, I do want to have fast implementations of gevent.httplib and gevent.wsgi in 0.14 that are at least as good as in 0.13. Not sure about the specifics yet.
> On Sat, May 14, 2011 at 5:27 PM, Jaime <jjja...@gmail.com> wrote:
> > Denis, do you have any plan to support fast, native wsgi when porting
> > to libev? Perhaps using third-party projects likehttps://github.com/jonashaag/bjoern > > which is also based on libev?
> Yes, I do want to have fast implementations of gevent.httplib and
> gevent.wsgi in 0.14 that are at least as
> good as in 0.13. Not sure about the specifics yet.
