Getting more information into Firebug

[mnot]

Hi,

One of the things I do in my day job is helping people develop and
debug HTTP-based APIs and services.

Often, because services are composed across several layers of caching,
aggregation, etc. and pull in things like databases and other
services, people want to get a "trace" view that shows the whole
picture, rather than just one component.

One of the ways to do this is to put the information into HTTP
response headers, so that downstream clients can consume it.

Making this sort of information visible in Firebug would be awesome.
E.g., if a server can supply information about how it's spent it's
time, the waterfall can give much more detailed information than just
'waiting'.

Upstream caches could expose more information to show whether it was a
hit, a miss, and so on. Some caches (e.g., Squid and Traffic Server)
already make this information available in proprietary format.

Back-end servers how the response was generated, and with a bit more
work, can help map out where the components of a response were drawn
from, with the overhead of each made visible.

Would folks at Firebug be interested in working on this?

It would mean defining a few new response headers that describe the
trace information, along with the appropriate UI changes. It would
probably also require supporting HTTP trailers for those headers,
since this sort of information is often not available until after the
response is generated.

Cheers,

[Honza (Jan Odvarko)]

Yes, great, this is something I have been also thinking about, but
didn't have enough knowledge what time-profiling techniques/
technologies are available on the server side.

Here are some comments/notes.

- response headers
yes, I think this is the way how to get the server side timings to the
client.


- 'waiting' phase
Precisely, this phase would be divided into more sections describing
server side timings info. I don't know what are the options here and
it could also depends on the server side capabilities.


- HTTP Archive Format
I believe we can also extend the HAR format to cover also the server
side timing info. The question is whether we can define solid
structure for gathered timings shared across various servers.

http://groups.google.com/group/http-archive-specification/web/har-1-2-spec?hl=en


- NetExport
This is an extension to Firebug that allows exporting the Net panel
data
http://www.softwareishard.com/blog/netexport/

As soon as I have the server-timings in a response headers, I can
extend also the waterfall diagram to display it. I think the code
shouldn't be directly part of Firebug, but provided through an
extension.


- HAR Viewer
Since the purpose of the HAR format is to export HTTP tracing data
from various tools in the same format, there is also an online viewer
that allows to quicly preview & validate existing logs.
http://www.softwareishard.com/blog/netexport/

In case we have a solid extension to the HAR spec, the viewer could be
adapted too.

Honza

[Don Dwiggins]

This would be useful for my application as well. Maybe it'll help your
thinking to have a specific example:

I have a web app implemented in ExtJS; the web server acts primarily as
a dispatcher to various backend servers (based on domain name); each
backend server accesses the database. It would be useful to have the
backend server provide timings for database access and internal request
processing, and have them passed up to the web server, where it'd be
included in the response headers as you describe. Since I'm using
XMLRPC between the web server and backend server, I'd want to instrument
the time taken to encode and decode the data as well.

> that allows to quicly preview& validate existing logs.

--

Don Dwiggins
Advanced Publishing Technology

[Honza (Jan Odvarko)]

I have created a bug report for this:
http://code.google.com/p/fbug/issues/detail?id=3613

Honza

> >http://groups.google.com/group/http-archive-specification/web/har-1-2...