EventMachine performance Linux vs. Mac OS X

[Christoph Pilka]

Hi folks,

I've deployed a web crawler which I've built on top of em-synchrony. I
can run 400 req/s on my tiny MacBook Air and get 30 MBit/s (50 MBit
ADSL line). I've put the same code on a Debian Box and the maximum is
70 req/s and 7 MBit/s (100 MBit server). Anyone here has a hint for me
what could cause such a difference? Is it OS X with kqueue over Linux
with select/epoll? Should I try to run the code on a FreeBSD box cause
of kqueue? Which parameters would you tweak on a linux box? I've
increased all ulimit params but nothing changed :(

Cheerio,
Chris

[Aman Gupta]

Are you using EM.epoll=true and EM 1.0.0.beta release?

> --
> You received this message because you are subscribed to the Google Groups "EventMachine" group.
> To post to this group, send email to eventm...@googlegroups.com.
> To unsubscribe from this group, send email to eventmachine...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/eventmachine?hl=en.
>

[Christoph Pilka]

Yepp, both. Eventmachine 1.0.0.beta.4 and EM.poll=true

[Paul Makepeace]

Dumb questions… are these machines on the same network? Connected the
same way (wifi, eth, …)? Using the same (DNS) resolvers? Crawling the
same pages? Connecting through the same (trans)proxies, if applicable?
Where is the data from the crawl being written? (Airs have SSDs)

Paul

> Cheerio,
> Chris