How Parallel is Chrome?

[Alex Wheeler]

I am creating a code which creates thumbnails of webpages using nodejs and the debugging interface into Chromium.  This is pretty much complete now. The project runs on a 1Gbps link.and a very fast machine.  I'm looking for multiple tabs to be opening and rendering pages simultaneously.  This could be up to 50 pages at once.

I would like to ask the experts here if there are  limitations are on Chromium relating to parallel tasks? 

I ran a quick manual test opening 20 tabs when launching chrome. and saw that only 6 tabs at once were loading.  I also saw issues with that if the active tab is loading, then the other tabs are blocked from loading, or at least prioritize.

I would like to get somekind of plan together on what I will need to change to make this work how I need it too.  Any suggestions or insights would be really appreciated.

Thanks

Alex

[William Chan (陈智昌)]

The question of how parallel Chromium is is too vague to answer :) There are a variety of limitations. Let me throw out some:

* If you share renderer processes, then one tab can block another tab if it monopolizes the main thread (or the IPC thread for that matter)

* I think you mentioned elsewhere you were using an extension. That extension could be the bottleneck for all your loading, if the webRequest API handlers aren't responding quickly enough.

* There are networking limits, like connections per host and what not.

Identifying all the various parallelization limits would be a PITA. In order to figure out what's blocking progress for you, I recommend using a debugger, about:tracing, and task manager (to see if any process is CPU bottlenecked), and perhaps about:net-internals if you think it's a networking issue.

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev

[Alex]

Hi,

Thanks for the answer.  It was more of a general question to get a feel.  After doing some recent testing and looking at net-internals, it's pretty clear i'm running out of resources pretty quickly when opening up a socket.  I was loading 20 tabs simultaneously.  Chrome doesn't like this very much.  There is a hard limit of 256 sockets which can get chewed up pretty fast, especially when the old sockets hang around idle for some time.

I think my way forwards will be to create around 10 separate chrome instances each running under a different username.  Then have each one loading around 5->10 tabs at a time.  I will set out a test strategy to see what works best.

Going back to the hard limit of 256 sockets.  Please can I ask, what is the idle timeout for these?  They don't seem to close immediately when I close a window.  Is there an easy way I could enforce that or at least issue a command that could flush the sockets for a particular tab/window before I close it?  What would be the impact if I increased these sockets to something larger and recompiled?  Would it be a simple case of one variable to change in the code?

Thanks

Alex

[Matthew Menke]

Idle timeout for never sockets is 10 seconds.  For sockets that have been used, it's a couple minutes (4?  5?).  If we're at the 256 socket limit, and a new request comes in that needs a socket, and we have an idle socket, we go ahead and close that idle socket, so idle sockets should not slow down requests that need new sockets, other than perhaps having some effect on preconnect.

Sockets are not associated with a tab - if you have 5 windows open for the same site, or 5 different pages have content from facebook, they'll be shared.  Network stack basically doesn't even know what a tab is.  There's just one value to change the limit (Note that we have a lower limit when using a proxy, and only allow 6 connections per domain...And there's a DNS parallelism limit, and SPDY has a limit per session.  Can't remember any other limits, at the moment).