Chrome Histograms

[Mussa Kiroga]

Hello,

I have a test whose verification is a histogram bucket and its value. Currently, the test navigates to chrome://histograms/<histogram_name> and parses the text.

Is there a better way to do this ? Specifically, we would like to free the test from having to load the UI (Chrome), could there be a file that can be read directly ? 

Thanks,

Mussa

[Daniel Erat]

Is the test's purpose to verify that a histogram is being sent, or is the histogram just being used as a stand-in for whatever's actually being tested?

Metrics are sent by Chrome, so if you actually want to verify them, I'm not sure what else you could do short of hacking up the metrics library to write them somewhere else instead of passing them to the browser.

--
--
Chromium OS Developers mailing list: chromiu...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-os-dev?hl=en

[Mussa Kiroga]

Seond option : Histogram is being used as a stand-in: the thing being tested publishes its result as a histogram with a specific name ....

[Daniel Erat]

(Are you testing a confidential or unannounced feature, or can more details be provided?)

Does the thing being tested produce other side effects that you could observe instead of the histogram? It's best to test that the code is actually working as opposed to just reporting that it's working.

[Mussa Kiroga]

More details can be provided. An example would be :

chrome://histograms/Media.GpuVideoDecoderInitializeStatus

I just imagined, whatever information is on that page existed somewhere else, sort of like an xml file...

I will in parallel ask the developers in question if this can be done the way you are suggesting.

[Luigi Semenzato]

I think one can get that information directly from javascript.  Here's a snippet of code I wrote that copies some functionality from the telemetry testing system.  It may be enough to figure out how to do it.

    def get_histogram(self, histogram_function, histogram_name):
        """Returns the requested histogram as a python object."""
        # TODO(semenzato): Ideally this function should know if the histogram
        # is browser or renderer from the histogram name.  Maybe it could try
        # both, and remember the successful one.
        assert histogram_function in ("getHistogram", "getBrowserHistogram")
        js = 'statsCollectionController.%s("%s")' % \
            (histogram_function, histogram_name)
        result = self.eval_js(js)
        return json.loads(js_return_value(result))

I am not sure if this works on regular Chrome or if it needs some special flags.  This is how it works for me:

CHROME_FLAGS = \

  ",".join((

            "--disable-component-extensions-with-background-pages",

            "--disable-default-apps",

            "--disable-gaia-services"

            "--enable-gpu-benchmarking",

            "--enable-net-benchmarking",

            "--enable-per-tile-painting",

            "--enable-smooth-scrolling",

            "--enable-stats-collection-bindings",

            "--enable-threaded-compositing",

            "--ignore-certificate-errors",

            "--metrics-recording-only",

            "--no-default-browser-check",

            "--no-first-run",

            "--no-proxy-server",

            "--oobe-skip-postlogin",

            "--remote-debugging-port=9222",

            "--start-maximized",

            "--user-agent=" + USER_AGENT,

            "--vmodule=*/chromeos/net/*=2\\,*/chromeos/login/*=2",

            ))

[Mussa Kiroga]

Thanks Gayane. I'm writing end to end tests using autotest in python.

My test:

1. Play a video

2. check that hardware decoding was used by going to chrome://histograms/Media.GpuVideoDecoderInitializeStatus

My Goal:

Step 2 to avoid having to load up a tab just so that to get the histogram values.

[Mussa Kiroga]

Luigi - What is/are the histogram (s) that you use this for. I'm sure, it will work on any, just want to see an example.

[Luigi Semenzato]

This is how I use it:

                histogram = self.get_histogram("getBrowserHistogram",

                                               "Tabs.Discard.DiscardCount")

Some more info:

    def eval_js(self, expr):

        """Evaluates JS expression EXPR."""

        json = \

          {"params": {"returnByValue": True,

                      "expression": expr, },

           "method": "Runtime.evaluate",

          }

        return self.ws_request(json)

    def ws_request(self, request):

        """Sends a request on a websocket."""

        request['id'] = self._request_id

        request_string = json.dumps(request)

        logging.warning(("WS REQUEST: %s" % request_string))

        while True:

            result = self._websocket.send(request_string)

            response_string = self._websocket.recv()

            logging.warning(("WS RESPONSE: %s" % response_string))

            response_json = json.loads(response_string)

            if 'id' in response_json and \

                response_json['id'] == self._request_id:

                break

            # self._notification_queue.append(response_json)

        self._request_id += 1

        return response_json

The websocket, IIRC, is the remote debugging socket for the Chrome remote debugging protocol.

[Luigi Semenzato]

I should say, though, that this pretty much a last-resort solution.  There are a number of other side effects from playing a video correctly that would be more meaningful, as Dan said.

[Daniel Erat]

(Suggestion made over IM: play a solid-color video file, then take a screenshot and check it.)

[Achuith Bhandarkar]

I believe you just need to pass '--enable-stats-collection-bindings' as extra_browser_args into chrome.Chrome(), and you should be able to use the methods in histogram_utils with the video tab.