Suggestions for HAR 1.3
Jan Honza Odvarko
2) Post data encoding
3) Root resource
4) Navigation Timing
5) Geo Location
6) Page screenshots
7) Connection speed
===
#1) Errors
Thread: http://groups.google.com/group/http-archive-specification/browse_thread/thread/4d6dc706200761cf?hl=en
There can be many different errors (DNS, interrupted connection, browser problem, socket or proxy problem, etc.), corresponding field should be generic enough to cover them all. Since an error is (always?) related to a request the field should be part of a |request| object.
Could be a free text field or an object with detailed info.
"request": {
"error": {
"code": 1,
"description": "DNS",
"comment": ""
- error [object, optional]: Error object with detailed error info
- error.code [number]: Error identifier
- error.description [text, optional]: Error description
- error.comment [text, optional]: Standard comment field
We need some examples that proves the suggestion.
#2) Post data encoding
Thread: https://groups.google.com/d/topic/http-archive-specification/dJBMXTEAGG4/discussion
Binary posted data should have an encoding specified (just like for binary response bodies).
"request": {
"postData": {
"encoding": "base64",
...
encoding [text, optional]: Encoding used for postData.text field e.g "base64". Leave out this field if the text field encoded from its original character set into UTF-8.
#3) Root resource
Thread: http://groups.google.com/group/http-archive-specification/browse_thread/thread/84945ac2eceab3e7?hl=en
However, not sure if this field is really necessary
"pages": [
{
"requestedURL": "",
...
requestedURL [text, optional]: The first (root) resource URL originally requested by the user.
#4) Navigation Timing
Spec: http://w3c-test.org/webperf/specs/NavigationTiming/
Thread: http://groups.google.com/group/http-archive-specification/browse_thread/thread/9787363d488403a7?hl=en
There is a lot of new timing fields (also overlapping |request.timings| object).
Perhaps this would deserve whole new object?
"request": {
"performanceTiming": { ... }
}
performanceTiming [object, optional]: Detailed timing-related information (see NavigationTiming spec).
===
Other things that didn't make it into 1.3
#5) Geo Location
Geographic location included in HAR. There should be only one geographic location per page (?) so, this could be part of the
existing <page> element. (could be also part of the <creator> or <browser>?)
"pages": [
{
"geoLocation": "",
...
#6) Page screenshots
It should be possible to store also a list of page screenshots (taken at various phases of page load).
New field: screenshots in page (list of screenshots)
"pages": [
{
"screenshots": [
{
"data": "..."
"time": 2500
}
],
...
- data [string] - Encoded screenshot data (base64).
- time [string] - Number of milliseconds since the page load started (page.startedDataTime)
#7) Connection speed
Connection speed at the time of measurement. I believe that this value is related to a page load speed and, should be part of the page element.
"pages": [
{
"bitrate": 1024,
...
- bitrate [number, optional] - Connection speed (bit/s, bits per second).
===
I am personally voting at least for #1 and #2
Thoughts?
Honza
Sebo
#1) Errors
Thread: http://groups.google.com/group/http-archive-specification/browse_thread/thread/4d6dc706200761cf?hl=en
There can be many different errors (DNS, interrupted connection, browser problem, socket or proxy problem, etc.), corresponding field should be generic enough to cover them all. Since an error is (always?) related to a request the field should be part of a |request| object.
Could be a free text field or an object with detailed info.
"request": {
"error": {
"code": 1,
"description": "DNS",
"comment": ""}}
- error [object, optional]: Error object with detailed error info
- error.code [number]: Error identifier
- error.description [text, optional]: Error description
- error.comment [text, optional]: Standard comment field
We need some examples that proves the suggestion.
#2) Post data encoding
Thread: https://groups.google.com/d/topic/http-archive-specification/dJBMXTEAGG4/discussion
Binary posted data should have an encoding specified (just like for binary response bodies).
"request": {
"postData": {
"encoding": "base64",
...}}
encoding [text, optional]: Encoding used for postData.text field e.g "base64". Leave out this field if the text field encoded from its original character set into UTF-8.
#3) Root resource
Thread: http://groups.google.com/group/http-archive-specification/browse_thread/thread/84945ac2eceab3e7?hl=en
However, not sure if this field is really necessary
"pages": [
{
"requestedURL": "",
...}]
requestedURL [text, optional]: The first (root) resource URL originally requested by the user.
Another suggestion related to this:
Each request should hold the information about its origin. E.g. in Webkit this info is shown in the Initiator column. So it can be either the user, the requested HTML page, an AJAX call etc., that is causing the request. See also issue 1772 of the Firebug issue tracker.
#4) Navigation Timing
Spec: http://w3c-test.org/webperf/specs/NavigationTiming/
Thread: http://groups.google.com/group/http-archive-specification/browse_thread/thread/9787363d488403a7?hl=en
There is a lot of new timing fields (also overlapping |request.timings| object).
Perhaps this would deserve whole new object?
"request": {
"performanceTiming": { ... }
}
performanceTiming [object, optional]: Detailed timing-related information (see NavigationTiming spec).
Is that something, that could also make it into Firebug?
===
Other things that didn't make it into 1.3
#5) Geo Location
Geographic location included in HAR. There should be only one geographic location per page (?) so, this could be part of the
existing <page> element. (could be also part of the <creator> or <browser>?)
"pages": [
{
"geoLocation": "",
...}]
geoLocation [string, optional] - Geographical location of the client.
Also I don't think, that there's just one geographical information per page, because that's especially something for mobile devices. Imagine a script, that checks the location of the user every five seconds calling navigator.geolocation.getCurrentPosition(success, error);.
#6) Page screenshots
It should be possible to store also a list of page screenshots (taken at various phases of page load).
New field: screenshots in page (list of screenshots)
"pages": [
{
"screenshots": [
{
"data": "..."
"time": 2500
}
],
...}]
- data [string] - Encoded screenshot data (base64).
- time [string] - Number of milliseconds since the page load started (page.startedDataTime)
#7) Connection speed
Connection speed at the time of measurement. I believe that this value is related to a page load speed and, should be part of the page element.
"pages": [
{
"bitrate": 1024,
...}]
- bitrate [number, optional] - Connection speed (bit/s, bits per second).
Sebastian
Ricardo Oliveira
That's great that you organized this! im not entirely sure what's the criteria to include stuff in HAR, if it's 1) a nice piece of info to have independently of the way we get it or 2) something that can be readily measurable and available from existing measurements/browser mechanisms (low hanging fruit) or both 1) and 2).. some comments bellow:
> #1) Errors
> Thread: http://groups.google.com/group/http-archive-specification/browse_thread/thread/4d6dc706200761cf?hl=en
>
> There can be many different errors (DNS, interrupted connection, browser problem, socket or proxy problem, etc.), corresponding field should be generic enough to cover them all. Since an error is (always?) related to a request the field should be part of a |request| object.
>
> Could be a free text field or an object with detailed info.
>
> "request": {
> "error": {
> "code": 1,
> "description": "DNS",
> "comment": ""
> }
> }
>
> - error [object, optional]: Error object with detailed error info
> - error.code [number]: Error identifier
> - error.description [text, optional]: Error description
> - error.comment [text, optional]: Standard comment field
Even having just one field here (description) would be an imense boost to current HAR. People usually think web performance is all about speed, but correctness/availability is even more important.
I've included a failure page under http://irl.cs.ucla.edu/~rveloso/hartest/ with 2 diffferent failure modes: 1) a failed dns request and 2) a failed tcp connection establishment. A third case might be under way (where an already establish tcp connection gets dropped in the middle of the transfer). People can use the page as a test point to see what HAR gives in browsers like chrome/FF.
>
> We need some examples that proves the suggestion.
>
> #2) Post data encoding
> Thread: https://groups.google.com/d/topic/http-archive-specification/dJBMXTEAGG4/discussion
>
> Binary posted data should have an encoding specified (just like for binary response bodies).
>
> "request": {
> "postData": {
> "encoding": "base64",
> ...
> }
> }
>
> encoding [text, optional]: Encoding used for postData.text field e.g "base64". Leave out this field if the text field encoded from its original character set into UTF-8.
agreed
>
> #3) Root resource
> Thread: http://groups.google.com/group/http-archive-specification/browse_thread/thread/84945ac2eceab3e7?hl=en
>
> However, not sure if this field is really necessary
>
> "pages": [
> {
> "requestedURL": "",
> ...
> }
> ]
>
> requestedURL [text, optional]: The first (root) resource URL originally requested by the user.
would be a "nice to have"
>
> #4) Navigation Timing
> Spec: http://w3c-test.org/webperf/specs/NavigationTiming/
> Thread: http://groups.google.com/group/http-archive-specification/browse_thread/thread/9787363d488403a7?hl=en
>
> There is a lot of new timing fields (also overlapping |request.timings| object).
> Perhaps this would deserve whole new object?
>
> "request": {
> "performanceTiming": { ... }
> }
>
> performanceTiming [object, optional]: Detailed timing-related information (see NavigationTiming spec).
Seems to make sense at least to consolidate the timings given by w3c nav timings with the metrics seen in HAR (without duplicating metrics), even though right now nav timings and HAR values from FF/Chrome do not necessarily match. Vote for onStartRender as well.
>
> ===
> Other things that didn't make it into 1.3
>
> #5) Geo Location
> Geographic location included in HAR. There should be only one geographic location per page (?) so, this could be part of the
> existing <page> element. (could be also part of the <creator> or <browser>?)
>
> "pages": [
> {
> "geoLocation": "",
> ...
> }
> ]
>
> geoLocation [string, optional] - Geographical location of the client.
is this html 5 geolocation, and if so doesnt this require user level authorization ?
>
> #6) Page screenshots
> It should be possible to store also a list of page screenshots (taken at various phases of page load).
>
> New field: screenshots in page (list of screenshots)
>
> "pages": [
> {
> "screenshots": [
> {
> "data": "..."
> "time": 2500
> }
> ],
> ...
> }
> ]
>
> - data [string] - Encoded screenshot data (base64).
> - time [string] - Number of milliseconds since the page load started (page.startedDataTime)
for this to make sense there should already be some browser support to inject images into har, which afaik doesnt exist so far...
>
>
> #7) Connection speed
> Connection speed at the time of measurement. I believe that this value is related to a page load speed and, should be part of the page element.
>
> "pages": [
> {
> "bitrate": 1024,
> ...
> }
> ]
>
> - bitrate [number, optional] - Connection speed (bit/s, bits per second).
im not sure about this one, since the process to compute connection speed is tricky. modern browsers use multiple tcp connections to fetch objects in parallel from a same server, it's possible to approximate this, but it requires detection of overlaps in the waterfall. Also there can be cross traffic going on... would be nice to have here, but i have my doubts on how people would compute this.
Cheers
--Ricardo
Pieter Ennes
> 1) Errors
> 2) Post data encoding
> 3) Root resource
> 4) Navigation Timing
> 5) Geo Location
> 6) Page screenshots
> 7) Connection speed
>
> ===
>
> #1) Errors
> Thread:
> http://groups.google.com/group/http-archive-specification/browse_thread/thread/4d6dc706200761cf?hl=en
>
> There can be many different errors (DNS, interrupted connection,
> browser problem, socket or proxy problem, etc.),
It would be nice to have that subsystem classification in the error
object too, somehow.
Agree, but even better when it would allow for double wrappings: e.g.
gzip+base64 seems to make sense.
> #3) Root resource
> Thread:
> http://groups.google.com/group/http-archive-specification/browse_thread/thread/84945ac2eceab3e7?hl=en
>
> However, not sure if this field is really necessary
>
> "pages": [
> {
> "requestedURL": "",
> ...
> }
> ]
>
> requestedURL [text, optional]: The first (root) resource URL
> originally requested by the user.
>
> #4) Navigation Timing
> Spec: http://w3c-test.org/webperf/specs/NavigationTiming/
> Thread:
> http://groups.google.com/group/http-archive-specification/browse_thread/thread/9787363d488403a7?hl=en
>
> There is a lot of new timing fields (also overlapping |
> request.timings| object).
> Perhaps this would deserve whole new object?
Yes I think, the new object should be a direct copy from the W3C
fields.
> "request": {
> "performanceTiming": { ... }
> }
>
> performanceTiming [object, optional]: Detailed timing-related
> information (see NavigationTiming spec).
Also, the PerformanceNavigation object might be interesting to copy from
W3C in the same way.
> ===
> Other things that didn't make it into 1.3
>
> #5) Geo Location
> Geographic location included in HAR. There should be only one
> geographic location per page (?) so, this could be part of the
> existing <page> element. (could be also part of the <creator> or
> <browser>?)
>
> "pages": [
> {
> "geoLocation": "",
> ...
> }
> ]
>
> geoLocation [string, optional] - Geographical location of the client.
>
What does this string contain?
Would it make sense to store any of these in an object, to allow the
available level of granularity to be captured:
- human readable location
- lat/lng pair (+precision/radius?)
- geohash URI http://en.wikipedia.org/wiki/Geohash
- country code
- mobile MCC/MNC
I would vote to at least make lat/lng and country code machine readable
in the geoLocation object.
> #6) Page screenshots
> It should be possible to store also a list of page screenshots (taken
> at various phases of page load).
>
> New field: screenshots in page (list of screenshots)
>
> "pages": [
> {
> "screenshots": [
> {
> "data": "..."
> "time": 2500
> }
> ],
> ...
> }
> ]
>
> - data [string] - Encoded screenshot data (base64).
> - time [string] - Number of milliseconds since the page load started
> (page.startedDataTime)
Suggestions:
- "mimeType", to indicate image type
- "dataUrl", same as data, but then with screenshot available from a
remote URL
- "width", "height", width and height of the image (often known at
creation time, yet can help clients deciding on UI without decoding the
image)
- "scaleFactor", image scale ratio between 0 and 1 (to keep HAR size
minimal, scaling down screenshots seems like something that will be
often done)
- "progress", pageload percentage at time of snapshot as returned by the
browser
- "title", human readable title (to be used as 'alt' text)
>
> #7) Connection speed
> Connection speed at the time of measurement. I believe that this value
> is related to a page load speed and, should be part of the page
> element.
>
> "pages": [
> {
> "bitrate": 1024,
> ...
> }
> ]
>
> - bitrate [number, optional] - Connection speed (bit/s, bits per
> second).
Downstream? Upstream could be equally relevant, or more.
> ===
>
> I am personally voting at least for #1 and #2
Me too, and #6. #5 could use some more tinkering.
--
- Pieter
Drit
In regards of errors I would highly recommend we do not enforce some
error list, but keep it open. There is issues with browsers, tools,
and localization strings. Code/description/Comment are sufficient.
Other items I would love to see included:
# 6 - screenshots (and also add filmstrips/video - since most tools
have the two as separate features and might require different display)
# 8 Referer URL - the inclusion of third party tags that keep
including other tags, makes it necessary to have some easy way to see
the referer.
#9 Initiator - Referer URL is not sufficient as often calls are
triggered by JavaScript. Chrome and IE9 display a new column in their
Dev Tools>Network showing who initiated the request, and also provide
other details like line #. This is very helpful in debugging who
triggered a request- and since HAR is being used to communicated
across companies and departments, it will be great to include it. It
is duplicating the referer in a lot of cases, but they are 2 different
things and not all tools/browsers can provide Initiator at this time.
drit
On Nov 21, 3:51 am, Jan Honza Odvarko <odva...@gmail.com> wrote:
> 1) Errors
> 2) Post data encoding
> 3) Root resource
> 4) Navigation Timing
> 5) Geo Location
> 6) Page screenshots
> 7) Connection speed
>
> ===
>
> #1) Errors
> Thread:http://groups.google.com/group/http-archive-specification/browse_thre...
>
> There can be many different errors (DNS, interrupted connection, browser
> problem, socket or proxy problem, etc.), corresponding field should be
> generic enough to cover them all. Since an error is (always?) related to a
> request the field should be part of a |request| object.
>
> Could be a free text field or an object with detailed info.
>
> "request": {
> "error": {
> "code": 1,
> "description": "DNS",
> "comment": ""
> }
>
> }
>
> - error [object, optional]: Error object with detailed error info
> - error.code [number]: Error identifier
> - error.description [text, optional]: Error description
> - error.comment [text, optional]: Standard comment field
>
> We need some examples that proves the suggestion.
>
> #2) Post data encoding
> Thread:https://groups.google.com/d/topic/http-archive-specification/dJBMXTEA...
>
> Binary posted data should have an encoding specified (just like for binary
> response bodies).
>
> "request": {
> "postData": {
> "encoding": "base64",
> ...
> }
> }
> *
> *encoding [text, optional]: Encoding used for *postData.text* field e.g
> "base64". Leave out this field if the text field encoded from its original
> character set into UTF-8.
>
> #3) Root resource
> Thread:http://groups.google.com/group/http-archive-specification/browse_thre...
>
> However, not sure if this field is really necessary
>
> "pages": [
> {
> "requestedURL": "",
> ...
> }
> ]
> *
> *requestedURL [text, optional]: The first (root) resource URL originally
> requested by the user.
>
> #4) Navigation Timing
> Spec:http://w3c-test.org/webperf/specs/NavigationTiming/
> Thread:http://groups.google.com/group/http-archive-specification/browse_thre...
>
> There is a lot of new timing fields (also overlapping |request.timings|
> object).
> Perhaps this would deserve whole new object?
>
> "request": {
> "performanceTiming": { ... }}
>
> *
> *performanceTiming [object, optional]: Detailed timing-related information
> (see NavigationTiming spec).
>
> ===
> Other things that didn't make it into 1.3
>
> #5) Geo Location
> Geographic location included in HAR. There should be only one geographic
> location per page (?) so, this could be part of the
> existing <page> element. (could be also part of the <creator> or <browser>?)
>
> "pages": [
> {
> "geoLocation": "",
> ...
> }
> ]
> *
> *
> geoLocation [string, optional] - Geographical location of the client.
>
> #6) Page screenshots
> It should be possible to store also a list of page screenshots (taken at
> various phases of page load).
>
> New field: screenshots in page (list of screenshots)
>
> "pages": [
> {
> "screenshots": [
> {
> "data": "..."
> "time": 2500
> }
> ],
> ...
> }
> ]
> *
> *- data [string] - Encoded screenshot data (base64).
> - time [string] - Number of milliseconds since the page load started
> (page.startedDataTime)
>
> #7) Connection speed
> Connection speed at the time of measurement. I believe that this value is
> related to a page load speed and, should be part of the page element.
>
> "pages": [
> {
> "bitrate": 1024,
> ...
> }
> ]
> *
> *- bitrate [number, optional] - Connection speed (bit/s, bits per second).
Pat Meenan
Thanks,
-Pat