Adding logging to ETW for Chrome Developer Tools

[Hayden Livingston]

Hey Chromium Dev,

Is there interest in adding logging that is availabile from the Chrome Dev Tools to ETW on Windows? Internet Explorer has a wealth of ETW events that are logged. It's easy to then share that data with developers and testers to improve the page rendering quality or find issues.

I suppose I could send a code review to plumb the logging, but wanted to know if others have interest in this?

Hayden.

[Ryan Sleevi]

Given that DevTools already has this logging, is shareable, and most importantly, is cross-platform, what would be the benefit/value in logging via ETW?

>
> Hayden.
>
> --
> --
> Chromium Developers mailing list: chromi...@chromium.org
> View archives, change email options, or unsubscribe:
> http://groups.google.com/a/chromium.org/group/chromium-dev

[Hayden Livingston]

There is additional logging done by Windows and their components in ETW. For example, I wanted to see why Chrome was taking longer to route to our website, when on similar class machines, IE seemed unaffected. The ability to capture data from Windows and have it in that context is invaluable at times. In this case, I was able to narrow it down to the DNS resolver IE was using was different than Chrome. Only possible because I was able to follow the causality from IE to its DNS and see that Chrome was doing something else.

Sawbuck has some limited logging, but it's not that great.

I do get the cross-platform goodness of Chrome DevTools, but this is one of those things that the platform tooling goes a long way.

How easy is it to get stuff like this in Chromium? I suppose there's always a case to demonstrate, but how does one show you guys the value.

[Ryan Sleevi]

On Aug 8, 2014 12:03 AM, "Hayden Livingston" <halivi...@gmail.com> wrote:
>
> There is additional logging done by Windows and their components in ETW. For example, I wanted to see why Chrome was taking longer to route to our website, when on similar class machines, IE seemed unaffected. The ability to capture data from Windows and have it in that context is invaluable at times. In this case, I was able to narrow it down to the DNS resolver IE was using was different than Chrome. Only possible because I was able to follow the causality from IE to its DNS and see that Chrome was doing something else.
>
> Sawbuck has some limited logging, but it's not that great.
>

Right, this is part of why I was asking.

> I do get the cross-platform goodness of Chrome DevTools, but this is one of those things that the platform tooling goes a long way.
>
> How easy is it to get stuff like this in Chromium? I suppose there's always a case to demonstrate, but how does one show you guys the value.

Well, usually the good starting point is to compare with the existing implementation and identify what's missing.

Usually "another tool is easier" isn't a great case, but "this awesome thing isn't possible" is better.

I'm assuming you also looked at chrome://tracing, which is cross-platform and equally closer to some of those particular aspects of ETW? There's also work to surface additional data (from the network layer) up.

Usually, the questions relate to how much complexity is involved, how will it work on other platforms, are there better solutions to the problem, and is the technical debt worth the perceived rewards.

It's also important to keep perspective - when the code we ship goes to hundreds of millions of users, does this benefit them all (like, say, a new web platform feature)? Does it benefit a particular community (like DevTools does Web Developers)? Or is this a niche feature?

One signal used for that is stars on a bug, so if you still feel its useful, filing a bug and evangelizing it can help. If you're a developer looking to implement it, usually writing up a design doc, figuring out what would need to change, and sending it either to the OWNERS of the code affected by the design or, if broadly impacting, to chromium-dev (or one of the appropriate sub-team lists like net, media, GPU, etc), will help gauge receptiveness to the feature.

The bad path would be skipping all these and going straight to coding, which usually wastes your time and that of reviewers, who end up wanting to have that design discussion anyways :)

[Dan Sinclair]

+Etienne Bergeron

There is some ETW support in chrome://tracing. Adding Etienne who did a bunch of work around ETW support.

dan

[Hayden Livingston]

Sweet. If we could share the provider guids .. I searched into the source and couldn't find much other than the logging exposed via Sawbuck.

[Hayden Livingston]

Etienne? Any ideas? I can't seem to find details on this.

[Chris Hamilton]

Sorry, Etienne is on vacation until next week.

As far as I can remember, he actually wanted to *export* Chrome tracing events via ETW, for the same reasons you mentioned. However, there was resistance to doing this (as you've seen). Instead, he worked on the opposite approach: getting kernel ETW events *into* Chrome tracing, so everything would be visible in a single UI. I know he had at least prototyped this, as I remember seeing CPU switches / disk events / network / page faults in a demo he gave. I'm not sure what the status of the code is, however.

Chris

[Hayden Livingston]

It would be nice to get his changes! Ok, thanks. Does Chrome Tracing have an option to write to a file, or some thing?

I'm going to try to and prototype and put it out there in the world wide web, to see if anyone wants to use it.

[Dan Sinclair]

Tracing has a save button which will let you save a json file.

dan

[Chris Hamilton]

If you can wait until Etienne is back I'm sure he'll be happy to update you on the current the state of the world, and share any code he may have lying around.

[Hayden Livingston]

Yes, of course. Thanks!

[Hayden Livingston]

Ah, I see, so it is indeed importing things into Chrome Tracing, not the other way around, hmm.

I really wish we'd export to ETW as well -- now that I think about it, why would there be a pushback? I'm guessing there is already some sort of abstraction for logging, we could just add it there only for Windows?

On Tue, Aug 19, 2014 at 1:07 PM, Etienne Bergeron <etie...@google.com> wrote:

Sorry to be late, I was on vacation.

The ETW support is implemented into Chrome a year ago and events are sent to the JSon files. As this is a 20% (more a 5% project), up-to now only the scheduling events are exported.

The intern fdoray@ completed the missing work two weeks ago to parse the events into chrome-tracing project.

I think a roll-deps of chrome-tracing project is still needed (see with nduca@).

You need to run chrome as an administrator to have the rights to get system events.

Elevation was planned for a future version. Decoding more types of events is also part of a future version.

--

Etienne Bergeron

Syzygy team, Chrome

[Francois Pierre Doray]

There is a fair amount of work to do to update the CL that was written a year ago to allow all chrome://tracing events to be forwarded to ETW, but I would be really happy to work on it. Perhaps I could start by looking at what changed in the chrome://tracing code since then and write a new design document to explain how Chrome->ETW would be implemented.

It would be extremely useful to have a list of real use cases for this new functionnality in order to show them to the involved owners.

On Wed, Aug 20, 2014 at 2:22 PM, Etienne Bergeron <etie...@google.com> wrote:

 * Fixing fdoray email.

On Wed, Aug 20, 2014 at 2:20 PM, Etienne Bergeron <etie...@google.com> wrote:

+fdoray

fdoray@ wrote a CL* a year ago to get tracing events (chrome events) to ETW and use them for external analysis.

This is not hard to implement. But, the chrome tracing team doesn't see the advantage of exporting chrome events to ETW.

  [*] https://codereview.chromium.org/23934003/

[Hayden Livingston]

I took a look at this CL. It seems fairly easy to export the data out to ETW which is good, and Francois' change has given me some direction. Thank you.

To Etienne and Francois, the scenario is simple -- being able to trace dynamically without interacting with Chrome on what the browser is doing really helps identify things like what JS files were downloaded (which incidentally does go to ETW), but I'm looking for more, basically all the F12 developer tool bar data.

The reason to stress on ETW here is that when I'm asking testing some webpage, I want to also get contextual data like what the OS was doing, and ETW providers that. Additionally, some tooling on Windows ETW makes the analysis easier.

My major user scenario is Chrome on Windows, so I see it through that lens.

Francois -- you mention there is fair amount of work? Do you think we could quantify that? Has the Chrome Tracing system changed significantly? I'm going to tinker with this in my spare time. If you have pointers, great. I'm also happy to let you do this, if you have a reasonable timeframe :-)

[Francois Pierre Doray]

Thanks for the details on the use case.

I drafted a design document for this feature: https://docs.google.com/document/d/1QqlEoPCoPZ8TUcoNJgZ5k2K2YekkqNNqRX8e1wCv9cI/edit?usp=sharing Please add your comments. In particular, it would be really helpful to have strong arguments in the "Benefits of Chrome to ETW" section.

The main difference since my first CL is that there is now "complete" events instead of pairs of "begin" and "end" events. That's the main reason why I thought it would take a fair amount of work, but after looking at the code, I don't believe it will be to difficult to implement.

[codeincomplete]

Can you get the console.timeline/timelineEnd to emit the label argument as part of the ETW event data? This would enable custom ETW events for web developers interested in performance and crashes.