Notice that the network monitor (and time measurement) is based on
nsIObserver (http-on-modify-request+http-on-examine-response+http-on-
examine-cached-response events) and nsIWebProgressListener.
The trouble is that all the events are posted to the UI thread which
could be blocked. This means that javascript handler execution can be
unpredictable delayed and so, the timing isn't precise.
Before I file a new bug for this, I wanted to check whether there are
any other options how to solve this.
Any thoughts?
Honza
I believe we have a bug on this already, for what it's worth.
The only way I can see of solving your problem is to put the actual
timestamp in all the necko callbacks. That's not cheap, though, and
just moves the issue (e.g. there's no guarantee that the necko thread
getting to run corresponds to the data actually arriving off the network).
Furthermore, the times as reported right now do have a certain
correspondence with the actual thing that matters: when the renderer
gets to see the data.
So it seems to me like the right thing to decide here, first, is
precisely what we're trying to measure and why. The second thing to
decide is whether measuring that thing from inside the Gecko process is
the right approach...
-Boris
> So it seems to me like the right thing to decide here,
> first, is precisely what we're trying to measure and why.
The goal is to provide info for web developers that can be used to
analyze page load performance. This info should be composed from
detailed timing info for each network request made by the monitored
page.
The ideal timing info (for each request) should describe following
phases:
Blocking: time spent in a queue waiting for a network connection.
Connecting: time required to create a TCP connection (part of this can
be DNS Resolution time).
Sending: time required to send HTTP request.
Waiting: waiting for a response from the server.
Receiving: time required to read entire response from the server (and/
or time required to read from cache).
Firebug is currently using http-on-modify-request + http-on-examine-
response + nsIWebProgressListener.onStatusChange (all in javascript).
The current logic is something like as follows:
http-on-modify-request: start of the request.
STATUS_RESOLVING -> STATUS_CONNECTING_TO: DNS Resolution time
STATUS_CONNECTING_TO -> STATUS_CONNECTED_TO: connecting
STATUS_CONNECTED_TO -> STATUS_WAITING_FOR: sending + waiting
http-on-examine-response: response received
http-on-examine-response -> STATUS_RECEIVING_FROM: reading data from
the server.
NOtice that the nsIWebProgressListener is originally designed for the
progress bar so, not much convenient for page load time analysis. For
example, some events are not sent at all if the next status happens
"fast" enough, which makes precise measurement difficult (better APIs
would certainly help a lot). But this is currently the only way I
know.
> The second thing to decide is whether measuring that thing
> from inside the Gecko process is the right approach...
I guess not. What are the other possibilities?
Honza
OK, gotcha.
>> The second thing to decide is whether measuring that thing
>> from inside the Gecko process is the right approach...
> I guess not. What are the other possibilities?
Well, a packet sniffer would tell you everything after the "blocking"
part, I would think..
-Boris
There's two solutions I can think of.
1. Have necko record all of the above information. It's probably
something that'd have to be turned on on a per-channel basis to avoid
slowing down channels where this information is not used.
2. Have off-main-thread callbacks for all of the above events. These
callbacks can then measure time and return. It's extremely important
that the callback don't start calling into non-threadsafe code.
Again, these callbacks would probably have to be enabled on a
per-channel basis to avoid perf overhead.
/ Jonas
> 2. Have off-main-thread callbacks for all of the above
> events. These callbacks can then measure time and return.
> It's extremely important that the callback don't start
> calling into non-threadsafe code.
Yes, understand. I guess the only thing that should be possible here
is to share data (gathered in the callbacks) between the background
thread and the main/UI thread. Specifically, at some point Firebug
must be able to read all the timing data (at least once) and use it to
present the graphical timeline to the user. Could this be achieved
through postMessage?
Is it possible to register such callbacks in 3.5? Could DOM workers be
somehow useful here?
> Well, a packet sniffer would tell you everything after
> the "blocking" part, I would think..
When the request is initiated by the user, the http-on-modify-request
is sent. Since this event till the first byte is actually sent (caught
in reliable off-main-thread callback) can be called as "blocking". So,
as soon as I can trust the time when http-on-modify-request is sent,
the problem is solved.
Honza
I don't think any of what we're discussing here will be possible in 3.5.
It'll have to be new APIs added in the next cycle.
Don't think DOM workers will be helpful here. You can use separate
threads without using DOM workers. You just have to be careful about it.
/ Jonas
I would like to make sure there is a bug for this so, it isn't
forgotten here in the noise. Does anybody know whether there is an
existing bug for this? If not, I am wiling to file one.
Honza
There is no bug on this as far as I'm aware. But in order to get this
fixed I'd also recommend that you actively try to find an owner and make
sure that someone with the knowledge to fix it makes it a priority.
In general I think we need more of this, making changes to gecko to
allow for a more awesome Firebug. I feel like too often firebug ends up
having to make gross hacks in order to implement the features it wants
to, rather than making changes to gecko.
For example, I think firebug has some pretty gross hacks in order to see
when XMLHttpRequests are being used, whereas I could quite easily add
some explicit notifications from the XHR code that firebug could listen
to. If only someone had asked :)
/ Jonas
We prefer to describe them as ingenious leveraging of existing APIs ;-).
Until say, six months ago, I'd say we were still learning the platform.
I think we are now in a position to ask. We've already had a couple of
big wins, like nsITraceableChannel. Several others are in the pipeline,
like nsIEventListenerInfo, cache-related events, and
ChromeObjectWrappers. Honza can speak to the XHR example, but generally
I would say be biggest problems for Firebug are in two areas where
asking has not been very successful: error handling and jsd. But I'll
keep trying ...
jjb
As far as the Firebug's XHR spy is concerned, I'll put together all
info and
try to summarize useful API improvements that would help to Firebug.
I'll file another bug for this.
Honza