Test Target Question

Skip to first unread message

En Low

Jan 29, 2021, 12:34:54 PMJan 29
to infra-dev

Hi! I am reaching out to see if there was any documentation/someone could explain the analysis for which test targets are run for a CL (I do have the link to that part of the source code ). I am trying to get a better understanding of the GN build graph analysis and though process behind that.

En (she/her/hers)
Timezone: EST (PST +3)

Dirk Pranke

Jan 29, 2021, 3:24:17 PMJan 29
to En Low, infra-dev

Are there particular questions you have in mind?

In general, the heavy lifting is done by GN in the `gn analyze` command.

The MB tool is a wrapper/translation layer that invokes GN; there's a design spec and a user guide for it.

Beyond that, on the bots we have a list of exclusions that GN doesn't know how to handle; if your CL touches one of those, we'll bypass the analyze command and run all the tests.

Let me know what else you'd like to know.

-- Dirk

You received this message because you are subscribed to the Google Groups "infra-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to infra-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/infra-dev/9775a52d-2dfd-4e48-8144-86996b996f5dn%40chromium.org.

Dirk Pranke

Feb 5, 2021, 5:50:04 PMFeb 5
to En Low, infra-dev
re-adding infra-dev@ for the record ...

On Fri, Feb 5, 2021 at 8:10 AM En Low <Low...@microsoft.com> wrote:

Thank you for the materials – they were very helpful! These are still a few points of clarification that I have:

  1. Is there a database to obtain test statistics, like total number of tests run, how often specific tests are run, breakdown of tests by platforms, suite, etc.?
We have a Google-internal test result database, but it's not currently publicly accessible and I don't think we have any immediate plans to make it be accessible. It's conceivable we could explore making some of the data publicly available in some form, but we'd have to do so pretty carefully. 

If you have general questions that you're curious about now, that might be easier to answer as one-offs.
  1. Is there more specific documentation on skipping builds/tests when running another commit queue dry run? In learning about the testing and building process, I have heard that there is a lot of “skipping”.
Many tests are configured to skip some of the tests they could run on the bots. They can do so a number of different ways; see https://source.chromium.org/chromium/chromium/src/+/master:docs/testing/on_disabling_tests.md . We are slowly working on improving our documentation here and making things a bit more consistent.

I'm happy to answer other questions as they come up as well,

-- Dirk


Thank you,

You received this message because you are subscribed to a topic in the Google Groups "infra-dev" group.
To unsubscribe from this topic, visit https://groups.google.com/a/chromium.org/d/topic/infra-dev/Gag-z3LWCo4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to infra-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/infra-dev/CAEoffTCZN%2Bx9k1rDFYkxxueEeT6fFJxqX63GE592AB5G3Kc%3DFQ%40mail.gmail.com.


En Low

Feb 8, 2021, 1:41:29 PMFeb 8
to infra-dev, Dirk Pranke, infra-dev, En Low

I see – thanks for letting me know. Would it be possible to see any recent summary reports on testing data? I am interested in the impact of testing retry logics and other practices aimed at improving efficiency impact on the number of test runs.  

Thanks again, 

Dirk Pranke

Feb 9, 2021, 2:36:02 PMFeb 9
to En Low, infra-dev, En Low
I don't think we have any reports that really have that data in an easily accessible form, unfortunately. 

-- Dirk

Reply all
Reply to author
0 new messages