Clarification on timestamps

[Kenneth Shaw]

Hi All,

I am running into a problem with a problem with timestamps for the chromedp (https://github.com/knq/chromedp) project. Specifically, I'd appreciate it if someone could clarify the different types of timestamps that I've encountered in the protocol definitions.

As far as I can tell, there are 3 different types of timestamps at play, spread over roughly 4 different uses, namely: 

a) Network.Timestamp
b) Runtime.Timestamp

c) Page timestamps (not an actual declared type)

d) Other object parameters named "timestamp" for commands/events (for instance, those in the Input domain)

All of these seem to have a JSON type declared as "number". I'd like to clarify what the expected behavior and time resolution of these are, as it appears over time the definition has changed (I may be wrong about this).

1. Network.Timestamp, as per the definition has SECOND resolution since the UTC epoch, from: https://chromedevtools.github.io/devtools-protocol/tot/Network/#type-Timestamp

2. Runtime.Timestamp, as per the definition has MILLISECOND resolution since the UTC epoch, from: https://chromedevtools.github.io/devtools-protocol/tot/Runtime/#type-Timestamp

3. Page timestamps (the various object parameters named "timestamp") are more like "bootstamps" and have SECOND resolution since the BOOT epoch (no reference; deduced by examining actual wire data)

4. Other object parameters named "timestamp" appear to all have SECOND resolution since the UTC epoch, from: various Chrome domains

If someone could clarify if 1, 2, 3 above is true (or correct where my understanding is wrong), and if it would be "safe" to map instances of #4 to Network.Timestamp, I'd greatly appreciate it!

My reasoning for this is that I'd like fully type-safe protocol/wire data flowing between chrome/go [via chromedp] and want to ensure that the wireline code is "correct."

Thanks in advance!

-Ken

[Pavel Feldman]

Thanks for the summary. Let me try to rename and document things as they are, then we can look into fixing them. https://chromium-review.googlesource.com/c/560557/

[Kenneth Shaw]

Thanks, appreciate the clarification via the changeset.

I have one question, however. MonotonicTime is now declared as a
specific epoch since an arbitrary point in time. Is this set per
instance of MonotonicTime, is it set based on the domain (ie,
instances from Page and Network share different epochs). I ask because
I've seen different numbers, and not sure exactly how the monotonic
epoch is being chosen.

-Ken


On Thu, Jul 6, 2017 at 5:23 AM, 'Pavel Feldman' via
chrome-debugging-protocol <chrome-debug...@googlegroups.com>
wrote:

> Thanks for the summary. Let me try to rename and document things as they
> are, then we can look into fixing them.
> https://chromium-review.googlesource.com/c/560557/
>

> --
> You received this message because you are subscribed to the Google Groups
> "chrome-debugging-protocol" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to chrome-debugging-p...@googlegroups.com.
> To post to this group, send email to
> chrome-debug...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/chrome-debugging-protocol/64af112c-56b9-4c99-be09-6097e70a4d62%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Pavel Feldman]

On Sat, Jul 8, 2017 at 5:37 PM, Kenneth Shaw <ken...@gmail.com> wrote:

Thanks, appreciate the clarification via the changeset.

I have one question, however. MonotonicTime is now declared as a
specific epoch since an arbitrary point in time. Is this set per
instance of MonotonicTime, is it set based on the domain (ie,
instances from Page and Network share different epochs). I ask because
I've seen different numbers, and not sure exactly how the monotonic
epoch is being chosen.

Monotonic time is per machine (computer). So all monotonic times share the baseline.

 

-Ken


On Thu, Jul 6, 2017 at 5:23 AM, 'Pavel Feldman' via

chrome-debugging-protocol <chrome-debugging-protocol@googlegroups.com>

wrote:
> Thanks for the summary. Let me try to rename and document things as they
> are, then we can look into fixing them.
> https://chromium-review.googlesource.com/c/560557/
>

> --
> You received this message because you are subscribed to the Google Groups
> "chrome-debugging-protocol" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to chrome-debugging-protocol+unsub...@googlegroups.com.

> To post to this group, send email to

> chrome-debugging-protocol@googlegroups.com.

> To view this discussion on the web visit

[Kenneth Shaw]

Also, I noticed that the HeapProfiler.lastSeenObjectId event has a
parameter "timestamp" that is still declared as a number. Should this
also be changed to a specific type?

-Ken

On Tue, Jul 11, 2017 at 1:52 AM, Pavel Feldman <pfel...@google.com> wrote:
>
>
> On Sat, Jul 8, 2017 at 5:37 PM, Kenneth Shaw <ken...@gmail.com> wrote:
>>
>> Thanks, appreciate the clarification via the changeset.
>>
>> I have one question, however. MonotonicTime is now declared as a
>> specific epoch since an arbitrary point in time. Is this set per
>> instance of MonotonicTime, is it set based on the domain (ie,
>> instances from Page and Network share different epochs). I ask because
>> I've seen different numbers, and not sure exactly how the monotonic
>> epoch is being chosen.
>>
>
> Monotonic time is per machine (computer). So all monotonic times share the
> baseline.
>
>>
>> -Ken
>>
>>
>> On Thu, Jul 6, 2017 at 5:23 AM, 'Pavel Feldman' via

>> chrome-debugging-protocol <chrome-debug...@googlegroups.com>

>> wrote:
>> > Thanks for the summary. Let me try to rename and document things as they
>> > are, then we can look into fixing them.
>> > https://chromium-review.googlesource.com/c/560557/
>> >
>> > --
>> > You received this message because you are subscribed to the Google
>> > Groups
>> > "chrome-debugging-protocol" group.
>> > To unsubscribe from this group and stop receiving emails from it, send
>> > an

>> > email to chrome-debugging-p...@googlegroups.com.

>> > To post to this group, send email to

>> > chrome-debug...@googlegroups.com.

[Lee Wei Yeong]

Is there a way to determine exactly which "arbitrary point in the past" that Network.MonotonicTime is referenced from?

>> > email to chrome-debugging-protocol+unsub...@googlegroups.com.

[Andrey Kosyakov]

On Fri, May 29, 2020 at 11:17 AM Lee Wei Yeong <evan...@gmail.com> wrote:

Is there a way to determine exactly which "arbitrary point in the past" that Network.MonotonicTime is referenced from?

In a way, yes -- depending on how "exact" you expect it to be. Note there's wallTime field in addition to timestamp in the parameters of Network.requestWillBeSent, so you can relate the monotonic time to the wall time at that point and extrapolate it for different points in monotonic time. In general, though, please keep in mind that this will only be an approximation, since the wall time goes slightly differently compared to the monotonic time (e.g. due to NTP time adjustments) -- this is why monotonic time has been chosen for all time measurements in the protocol.

Best regards,

Andrey.

[Lee Wei Yeong]

Allow me to be more specific

• I'm on macOS 10.15.5, Python 3.8.2,
• running code I wrote to monitor events incl. 
• "Network.responseReceived" (for which I will trigger a "Network.getResponseBody"),
• as well as "Network.webSocketFrameReceived","Network.webSocketFrameSent" events

(c.f. documentation https://chromedevtools.github.io/devtools-protocol/tot/Network)

What I meant by my question was, is there some way to relate the Network.MonotonicTime field value that I was observing in my traffic, to some known reference point in history e.g. 01 Jan 1970 for epoch, 01 Jan 1601 for filetime, rather than some arbitrary point in the past that cannot be determined (I tried to guess whether it was related to my system boot/startup time, but it seems off by a couple of hours)

Under the hood, it is doing `clock_gettime(CLOCK_MONOTONIC)`?

I'm also aware of https://chromium.googlesource.com/chromium/src/+/aee80e88c92958c951dc21309f31a64724dbb856/third_party/WebKit/Source/wtf/CurrentTime.cpp#45

I read that Monotonic time was selected, because it will be non-decreasing, and unaffected by changes in the system date/time setting

Sample traffic I'm capturing (although this is after my own processing and selecting fields - not sure if the wallTime field you mentioned is in the original traffic, but I don't recall coming across that field):

{"method":"Network.webSocketFrameSent","res":{"mask":true,"opcode":1,"payload":"[\"{\\\"msg\\\":\\\"unsub\\\",\\\"id\\\":\\\"vo6W8onEZrJ4seRqH\\\"}\"]","time":63303.591296}}

{"method":"Network.webSocketFrameReceived","res":{"mask":false,"opcode":1,"payload":"a[\"{\\\"msg\\\":\\\"removed\\\",\\\"collection\\\":\\\"questions\\\",\\\"id\\\":\\\"axL2frirRc6LtKPZQ\\\"}\"]","time":63303.81045}}

[Andrey Kosyakov]

On Fri, May 29, 2020 at 2:36 PM Lee Wei Yeong <evan...@gmail.com> wrote:

Allow me to be more specific

• I'm on macOS 10.15.5, Python 3.8.2,
• running code I wrote to monitor events incl. 
• "Network.responseReceived" (for which I will trigger a "Network.getResponseBody"),
• as well as "Network.webSocketFrameReceived","Network.webSocketFrameSent" events

(c.f. documentation https://chromedevtools.github.io/devtools-protocol/tot/Network)

What I meant by my question was, is there some way to relate the Network.MonotonicTime field value that I was observing in my traffic, to some known reference point in history e.g. 01 Jan 1970 for epoch, 01 Jan 1601 for filetime, rather than some arbitrary point in the past that cannot be determined (I tried to guess whether it was related to my system boot/startup time, but it seems off by a couple of hours)

Yes, except that point is not really a constant -- you need to keep track of Network.requestWillBeSent and Network.webSocketWillSendHandshakeRequest events, then you can compute the monotonic time origin as

monotonicTimeOffset = event.params.wallTime - event.params.timestamp

then for each subsequent event, you get the event wall time as

wallTime = event.params.timestamp + monotonicTimeOffset

(wallTime is in seconds since January 1, 1970). We recommend tracking the offset individually per request (this is what the front-end does), although in reality these are likely to be the same, or only differ occasionally and by not much.

 

Under the hood, it is doing `clock_gettime(CLOCK_MONOTONIC)`?

Ultimately, yes.

 

I'm also aware of https://chromium.googlesource.com/chromium/src/+/aee80e88c92958c951dc21309f31a64724dbb856/third_party/WebKit/Source/wtf/CurrentTime.cpp#45

I read that Monotonic time was selected, because it will be non-decreasing, and unaffected by changes in the system date/time setting

That's correct.

 

Best regards,

Andrey.