Issue 345853 in chromium: Create a google-internal LEGO pageset that just goes to google.com with LEGO on

[chro...@googlecode.com]

Status: Available
Owner: ----
CC: esei...@chromium.org
Labels: OS-All Pri-2 Type-Bug Restrict-View-Google Cr-Tests-Telemetry

New issue 345853 by nd...@chromium.org: Create a google-internal LEGO
pageset that just goes to google.com with LEGO on
http://code.google.com/p/chromium/issues/detail?id=345853

Goal:
pageset that goes to google.com, and it succesfully loads in the lego
eperiment.

Challenge:
telemetry starts the browser with a clean profile when recording. To get to
LEGO you need to be logged in.

I have no idea how to solve this. I think GWS folks have figured how they
would do this. But we need to share knowledge, and figure out what to land.

--
You received this message because project filter rules were set up to
notify you.
You may adjust your notification preferences at:
https://code.google.com/hosting/settings

[chro...@googlecode.com]

Updates:
Labels: Hotlist-Jank-Infra

Comment #1 on issue 345853 by nd...@chromium.org: Create a google-internal

LEGO pageset that just goes to google.com with LEGO on
http://code.google.com/p/chromium/issues/detail?id=345853

(No comment was entered for this change.)

[chro...@googlecode.com]

Updates:
Cc: nedngu...@google.com

Comment #2 on issue 345853 by nd...@chromium.org: Create a google-internal

[chro...@googlecode.com]

Comment #3 on issue 345853 by nedngu...@google.com: Create a

google-internal LEGO pageset that just goes to google.com with LEGO on
http://code.google.com/p/chromium/issues/detail?id=345853

Our temporary solution is recording the pageset on desktop and run the
measurement on Mobile.

[chro...@googlecode.com]

Updates:
Cc: fmea...@chromium.org
Labels: -Pri-2 -Type-Bug Pri-1 Type-Feature

Comment #4 on issue 345853 by to...@chromium.org: Create a google-internal

LEGO pageset that just goes to google.com with LEGO on
http://code.google.com/p/chromium/issues/detail?id=345853

Fadi set up one for production GWS. Let's get a parallel one for Lego.

Fadi, would you be willing to take this on?

Nat, Regarding the login problem, won't credentials.json solve this for us?

Ned, can you please point us at the best demo to record for this purpse?

We are probably going to want to fix issue 342795 so that we can benchmark
ToT lego as well as a stable lego and track regressions in across both
Chrome and Lego.

[chro...@googlecode.com]

Comment #5 on issue 345853 by nedngu...@google.com: Create a

google-internal LEGO pageset that just goes to google.com with LEGO on
http://code.google.com/p/chromium/issues/detail?id=345853

Tony, for recording, you can use:
https://www.google.com/?e=LegoSearch::OptIn

[chro...@googlecode.com]

Updates:
Cc: dpra...@chromium.org to...@chromium.org

Comment #6 on issue 345853 by nd...@chromium.org: Create a google-internal

LEGO pageset that just goes to google.com with LEGO on
http://code.google.com/p/chromium/issues/detail?id=345853

One thing we could consider is instead of writing our own pageset, we have
LEGO be the first customer of Tony's
automatic-internal-apps-to-chrome-waterfall workflow. Tony, forgive me for
butchering the name of your concept...

[chro...@googlecode.com]

Comment #7 on issue 345853 by to...@chromium.org: Create a google-internal

LEGO pageset that just goes to google.com with LEGO on
http://code.google.com/p/chromium/issues/detail?id=345853

We still need the page set, it would just be that recording will run
continuously.

[chro...@googlecode.com]

Updates:
Owner: fmea...@chromium.org
Cc: ern...@chromium.org

Comment #8 on issue 345853 by fmea...@chromium.org: Create a

google-internal LEGO pageset that just goes to google.com with LEGO on
http://code.google.com/p/chromium/issues/detail?id=345853

I will work with Nat/Manfred to make it silk compatible as well.

[chro...@googlecode.com]

Comment #9 on issue 345853 by nd...@chromium.org: Create a google-internal

LEGO pageset that just goes to google.com with LEGO on
http://code.google.com/p/chromium/issues/detail?id=345853

I'd like to see us use the same pageset that the lego folks are creating,
instead of building our own. They are going to be maintaining one
continuously. I'd like to see us automatically just use their blessed one,
continuously...

[chro...@googlecode.com]

Comment #10 on issue 345853 by nedngu...@google.com: Create a

google-internal LEGO pageset that just goes to google.com with LEGO on
http://code.google.com/p/chromium/issues/detail?id=345853

Update: Samarth put up a Lego sandbox here:
https://sky-lego.sandbox.google.com/?e=LegoSearch::OptIn

[chro...@googlecode.com]

Updates:
Status: Assigned
Owner: dpra...@chromium.org

Comment #11 on issue 345853 by nd...@chromium.org: Create a google-internal

LEGO pageset that just goes to google.com with LEGO on
http://code.google.com/p/chromium/issues/detail?id=345853

Dirk is investigating.

[chro...@googlecode.com]

Updates:
Cc: d...@chromium.org

Comment #12 on issue 345853 by dpr...@chromium.org: Create a

google-internal LEGO pageset that just goes to google.com with LEGO on
http://code.google.com/p/chromium/issues/detail?id=345853

Sheesh, sorry for the long delay on this ...

I'm finally at the point where I'm ready and able to start wiring the repos
together. Basically the way things are going to work is as follows:

1) We'll identify subtrees in google3 that will contain page sets,
measurements, and other telemetry-related things. As part of this, for now,
we'll need to identify which tests specifically need to be recorded and
uploaded to cloud storage.

2) We'll clone those over into directories in the new perf-internal chrome
repo. At the moment we will need create a few __init__.py files on the fly
to make the telemetry scripts work, but that can be easily fixed once we
get a better sense that we're happy with the directory structures and
mappings.

3) That perf-internal repo will be pulled in via DEPS entries in
src-internal/DEPS and mapped onto src/tools/perf/internal , where chrome
devs will be able to run the tests just like the public ones.

Once this is all working manually, I'd suggest that this particular bug /
task is complete. We can and should file other bugs to document other tasks
in the system, like updating recordings, updating measurements, adding try
jobs, etc.

[chro...@googlecode.com]

Comment #13 on issue 345853 by bugdro...@chromium.org: Create a

google-internal LEGO pageset that just goes to google.com with LEGO on

http://code.google.com/p/chromium/issues/detail?id=345853#c13

------------------------------------------------------------------
r261676 | dpr...@chromium.org | 2014-04-04T04:48:24.133519Z

Changed paths:
M
http://src.chromium.org/viewvc/chrome/trunk/src/.gitignore?r1=261676&r2=261675&pathrev=261676

Add tools/perf/internal to .gitignore .

This needs to land in preparation for mapping internal perf tests
from src-internal onto the main tree.

R=to...@chromium.org, nd...@chromium.org, d...@chromium.org
BUG=345853

Review URL: https://codereview.chromium.org/225103003
-----------------------------------------------------------------

[chro...@googlecode.com]

Comment #14 on issue 345853 by dpr...@chromium.org: Create a

google-internal LEGO pageset that just goes to google.com with LEGO on
http://code.google.com/p/chromium/issues/detail?id=345853

Okay, I had all of the source control shenanigans working, and then we hit
some weird issues with windows, so that got backed out.

I have a new CL posted to re-enable pulling perf-internal for just linux
and android checkouts temporarily, which should largely make developers
productive (even though we'll need mac and win bots soon enough).

In the meantime, I've also dusted off a couple of the page sets that were
checked in to google3/experimental/gwshack, and chrome devs should be able
to run them locally.

The recordings pointed to the internal sandbox instance rather than
google.com, but otherwise I think is sufficient to prove the concept.

Once this all lands and is available locally in a Chrome checkout we might
want to consider closing this bug. I'll be happy to walk anyone through the
details as needed, but the basic overview is as documented here:

https://chrome-internal.googlesource.com/chrome/perf-internal.git/+/master/README

The next obvious things to do are to work w/ the google3 devs to be able to
use the standalone telemetry zip + some additional things to upload new
changes , and to get the bots stood up and running these tests.

[chro...@googlecode.com]

Updates:
Status: Started

Comment #15 on issue 345853 by dpr...@chromium.org: Create a

google-internal LEGO pageset that just goes to google.com with LEGO on
http://code.google.com/p/chromium/issues/detail?id=345853

(No comment was entered for this change.)

[chro...@googlecode.com]

Updates:
Cc: jjts...@google.com

Comment #16 on issue 345853 by dpr...@chromium.org: Create a

google-internal LEGO pageset that just goes to google.com with LEGO on
http://code.google.com/p/chromium/issues/detail?id=345853

Okay, this should now be live and working on tip-of-tree.

I'm gonna leave this bug open for a day or two more to make sure it sticks.

I have no idea if the lego pagesets I loaded are particularly useful or
relevant at this point, of course. Hopefully Justin and/or Ned can confirm
that they are or confirm that no one on Chrome should bother looking at
them ...

[chro...@googlecode.com]

Comment #17 on issue 345853 by nedngu...@google.com: Create a

google-internal LEGO pageset that just goes to google.com with LEGO on
http://code.google.com/p/chromium/issues/detail?id=345853

Hi Dirk,
All the actions in those page_set are relevant, but the compound_actions
may contain things like "mainthread_jank" which are just our measurement
hacks in side gwshack/../perf/.

At the moment, I think the biggest issue for us is that there aren't many
existing telemetry metrics that are useful to us as web app developers. A
page set isn't that useful without the measurements, so I think you
shouldn't waste time on gwshacks' page sets at the moment. If you want a
few runnable one for bootstrapping, I can point you to some.

[chro...@googlecode.com]

Comment #18 on issue 345853 by dpr...@chromium.org: Create a

google-internal LEGO pageset that just goes to google.com with LEGO on
http://code.google.com/p/chromium/issues/detail?id=345853

Hi Ned,

Yeah, I pulled the 'mainthread_jank' measurement over as part of the
proof-of-concept.

Whenever you have stuff that you think is better, let me know. In theory at
this point you should be able to upload and even land patches to the new
repo, though I'm sure I'll have to walk you through this at least in order
to discover what I've screwed up or missed :).

[chro...@googlecode.com]

Comment #19 on issue 345853 by nd...@chromium.org: Create a google-internal

LEGO pageset that just goes to google.com with LEGO on
http://code.google.com/p/chromium/issues/detail?id=345853

Do we have a clean workflow for creating the WPR files for these recordings?

@jjtswan/@nedngueyn, do we have a LEGO pageset that uses the timeline based
measurement and reports simply smoothness? We should have the basic
capabilities to do so landed...

[chro...@googlecode.com]

Comment #20 on issue 345853 by dpr...@chromium.org: Create a

google-internal LEGO pageset that just goes to google.com with LEGO on
http://code.google.com/p/chromium/issues/detail?id=345853

> Do we have a clean workflow for creating the WPR files for these
> recordings?

I suppose that depends on your definitions ... at the very least, it is
easy enough for anyone who has local access to the servers needed to run
the recording to create the recording.

There is a potential next step to enable devs who don't have easy access to
the servers to be able to write a test and have some other system produce
the recording. This is doable, but I'm unclear how important this is (happy
to discuss further).

There is also the obvious next step of keeping recordings up to date as the
services change. That's a separate set of tasks (crbug.com/342795).

[chro...@googlecode.com]

Comment #21 on issue 345853 by dpr...@chromium.org: Create a

google-internal LEGO pageset that just goes to google.com with LEGO on
http://code.google.com/p/chromium/issues/detail?id=345853

Looks to be sticking, plus or minus a couple of weird git auth issues that
have been reported to chrome-team.

Nat, are you okay w/ me closing this given our discussions?

[chro...@googlecode.com]

Comment #22 on issue 345853 by nd...@chromium.org: Create a google-internal

LEGO pageset that just goes to google.com with LEGO on
http://code.google.com/p/chromium/issues/detail?id=345853

Totally! Thanks for bringing me up to speed the other day.

[chro...@googlecode.com]

Updates:
Status: Fixed

Comment #23 on issue 345853 by dpr...@chromium.org: Create a

google-internal LEGO pageset that just goes to google.com with LEGO on
http://code.google.com/p/chromium/issues/detail?id=345853

(No comment was entered for this change.)