Re: Google Chrome Perf Lab owners

[Annie Sullivan]

+speed-metrics-dev, benchmarking-dev

The name for the benchmarking framework that uses wprgo is Telemetry. Sorry it's confusing with other things also called telemetry!

Can you list the names of some of the metrics you're seeing zeroed out?

On Thu, Jan 3, 2019 at 6:35 PM Bruce Dawson <bruce...@google.com> wrote:

Annie, are you a good person to answer this question about the Web Page Replay tests?

On Thu, Jan 3, 2019 at 3:26 PM Todd Reifsteck <todd...@microsoft.com> wrote:

By WPRGo tests, we mean various tests that depend on Web Page Replay.

https://github.com/catapult-project/catapult/tree/master/web_page_replay_go

 

One example are the Desktop-Load suite. They seem to emit empty CPU usage metrics when run on VMs.

 

Any idea why this might occur off the top of your head?

 

-Todd

 

From: Bruce Dawson <bruce...@google.com>
Sent: Thursday, January 3, 2019 2:14 PM
To: Todd Reifsteck <todd...@microsoft.com>
Cc: Mike Decker <mde...@microsoft.com>
Subject: Re: Google Chrome Perf Lab owners

 

What are the WPRGo tests?

 

Off-hand I would think that we would want to fix the tests, but I guess it depends on what exactly needs fixing.

 

I'm a good point of contact for most Chromium Windows performance issues - if nothing else I can help redirect you to the right person.

 

On Thu, Jan 3, 2019 at 2:06 PM Todd Reifsteck <todd...@microsoft.com> wrote:

Bruce,

As you saw, we are building a Performance Lab to ensure that changes that we are making are solid before we upstream them to Chromium. We are observing a lot of interesting things when we review how the tests behave on VMs or on specific types of hardware.

 

The specific issue we are observing at the moment is that all WPRGo tests seem not to output CPU metrics when run in a VM. We aren’t sure if this is “intentional” or an “accident”.

 

We are trying to decide whether to “fix” tests/Chromium to record the CPU metrics “properly” or whether we need to alter our test environment to utilize physical hardware.

 

Are there specific developers that are a good “first point of contact” for this type of questions?

 

-Todd

 

--

Bruce Dawson

--

Bruce Dawson

[Annie Sullivan]

Re-adding speed-metrics-dev after fixing posting permissions.

[Todd Reifsteck]

We did a bit more digging. We believe this may be due to how Chromium’s time calculation code makes a decision whether to use Windows QPC or not.

 

We’ll share more after we experiment in a week or two.

 

-Todd

[Bruce Dawson]

That would make sense. If the times in question were very short (a few ms) then using the tick counter could easily lead to the benchmarks being reported as completely instantaneously.

Our method of deciding whether to use QPC is incorrect, I think, and maybe misguided, but we call QPC frequently enough that we don't want to pay the cost of an "expensive" version of it.

--

Bruce Dawson

[Todd Reifsteck]

Pardon the delay in communication! Will update this thread when we have results.

 

We are taking a look to see whether the mitigation from Rick James, https://chromium-review.googlesource.com/c/chromium/src/+/1413055, will enable this for us by allowing the tests to detect themselves as having sufficiently high precision when executed from VMs.

 

-Todd

[Todd Reifsteck]

We have confirmed that Rick’s fix improved the resolution of the metrics we were observing issues on when running certain tests.

 

One example is the speedometer benchmark where we previously saw all results rounded to the nearest millisecond and they are now showing resolution down to the microsecond.

 

-Todd

--
You received this message because you are subscribed to the Google Groups "Chrome Speed Metrics" group.
To unsubscribe from this group and stop receiving emails from it, send an email to speed-metrics-...@chromium.org.
To post to this group, send email to speed-me...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/speed-metrics-dev/BYAPR21MB11749568F4D4B3D4DDBD67D8C2970%40BYAPR21MB1174.namprd21.prod.outlook.com.