Rebaseline bot?

[Dominik Röttsches]

Hi Joel, others,

thanks for running the rebaseline bot. Is the bot currently active, after the Blink/Chromium merge? I've marked a few tests NeedsRebaseline but haven't seen the rebaseline commit msgs in the bugs yet.

For example:

https://codereview.chromium.org/1361953002

https://codereview.chromium.org/1363893002/

Thanks for your help,

Dominik

[Steve Kobes]

I found http://crbug.com/535351 which seems to track this.

Hopefully it can be fixed soon; a large backlog of rebaselines increases the risk of regressions going unnoticed (since a change may break a test that was previously marked for rebaseline).

[Dirk Pranke]

Yup, the bot has been inactive since the merge (it checks based on the svn revision, which is currently frozen :).

We have a fix in review; I expect it'll land today.

-- Dirk

[Walter Korman]

Is https://codereview.chromium.org/1379493002 the fix? It looks like it passed the CQ dry run and is awaiting review. I only press on this to speed it along as feasible, sharing Dominik's concern re: regression risk.

To unsubscribe from this group and stop receiving emails from it, send an email to blink-dev+...@chromium.org.

[Dominik Röttsches]

Ojan, Dirk, Joel,

On Thu, Oct 1, 2015 at 7:03 AM, Walter Korman <wko...@google.com> wrote:

Is https://codereview.chromium.org/1379493002 the fix?

Thanks everyone for towards landing this.

Dominik

[Joel Ong]

Sorry about the delay, I'm working on an additional fix to get the rebaseline bot up and running.

[Joel Ong]

The rebaseline bot is now fixed and has processed all pending rebaselines. Thanks for your patience everyone!

[Walter Korman]

Thank you for getting it operational again!

I am checking in as the last auto-rebaseline I see was on Oct 2, is it perhaps stuck? I know there is separate work planned to make it more robust to periodic failure per http://crbug.com/519124.

[Joel Ong]

Restarted it, let me know if it goes down again. I plan to get the buildbot version working which would eliminate the problem since it runs auto-rebaseline as a cron job.

[Stephen White]

It seems to be stuck again. Last ARB run was Tuesday at 00:19 EDT.

Thanks,

Stephen

[Joel Ong]

It's up again. It was running fine yesterday but not rebaselining due to Win10 failures. http://blinkrebaseline.appspot.com/?before=2015-10-07T06:00:00Z&numlogs=100

[Walter Korman]

It may be down again, I submitted a change yesterday and no rebaseline yet, and only one the day before.

[Joel Ong]

it appears to have been stuck on one particular rebaseline:

webkitpy.tool.commands.rebaseline: [INFO] INFO:root:upload.RealMain(['upload', '--assume_yes', '--server', 'https://codereview.chromium.org', '--message', 'Auto-rebaseline for r353763\n\nhttps://chromium.googlesource.com/chromium/src/+/df1458a0b4\n\nBUG=404597\nTBR=Emil', '--reviewers=Emil', '--cc', 'chromium...@chromium.org,blink-...@chromium.org', '--git_similarity', '50', '--base_url', 'https://chromium.googlesource.com/chromium/src.git@master', '--target_ref', 'refs/pending/heads/master', '--project', 'chromium', 'ac4ebe021dd185d7a86e930a623891619a415f1d', 'HEAD'])

webkitpy.tool.commands.rebaseline: [INFO] Presubmit checks took 2.3s to calculate.

webkitpy.tool.commands.rebaseline: [INFO] Presubmit checks passed.

webkitpy.tool.commands.rebaseline: [INFO] Upload server: https://codereview.chromium.org (change with -s/--server)

webkitpy.tool.commands.rebaseline: [INFO] Issue creation errors: {'reviewers': [u'Unknown user: Emil']}

webkitpy.tool.commands.rebaseline: [INFO] Got exception while uploading -- saving description to /usr/local/google/home/joelo/.git_cl_description_backup

[Dirk Pranke]

We might need to manually rebaseline those particular tests and then remove the rebaseline entries so that we're not stuck.

-- Dirk

[Levi Weintraub]

I think Emil changed his name in his git config (we were chatting about it at least). Seems like a bug in the tool that it can't find him now?

[Joel Ong]

I'll resubmit the lines for those tests, that should get the tool going again.

[Ojan Vafai]

The way it finds your email address is to read the email out of git blame. Didn't realize that the blame didn't always have your email address. :( 

You wouldn't want the tool to commit in this case because then the original author won't see the rebaseline patch land. We'll need to find some other way of getting email addresses out of git.

Looks like this is really easy to fix. We just need to add --show-email to the git blame call. Joel, mind fixing? (go ahead and resubmit the lines and then we can fix the issue asynchronously in a more leisurely fashion)

[Joel Ong]

SGTM

[Walter Korman]

I'm wrasslin' with some conflicting expectations now, is the bot processing them currently? The most recent one I see is 5am today, and the log shows:

http://blinkrebaseline.appspot.com/?before=2015-10-16T17:55:53Z&numlogs=3

as the most recent non-#QUIT, though admittedly off by ~12h in the timestamp for some reason, and it ends with:

webkitpy.tool.commands.rebaseline: [INFO] Issue creation errors: {'reviewers': [u'Unknown user: fmalita']}

webkitpy.tool.commands.rebaseline: [INFO] Got exception while uploading -- saving description to /usr/local/google/home/joelo/.git_cl_description_backup

[Florin Malita]

Looks like the same issue blocking earlier rebaselines due to unknown user 'Emil'.  Some commits started showing

Author: fmalita <fma...@chromium.org>

instead of previous

Author: fma...@chromium.org <fma...@chromium.org>

which I presume is why git blame doesn't show the full email address by default.

Sounds like Ojan's proposed fix should take care of it, but not sure what the status is.  Joel?

[Joel Ong]

I'm on it.

[Emil A Eklund]

...and it looks like it's down again.

[Ojan Vafai]

The root bug hasn't been fixed, so every time you commit a NeedsRebaseline patch it stalls. I assume Joel will fix this on Monday, but if someone wants to fix this before he can get to it, it's almost certainly a 1 line patch. You'd want to test manually to be sure.

https://code.google.com/p/chromium/codesearch#chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/tool/commands/rebaseline.py&q=rebaseline-o-matic&sq=package:chromium&type=cs&l=706 is where we read the blame of the file. That blame call needs to pass --show-email.

I *think* all you need to do is add --show-email to the command in https://code.google.com/p/chromium/codesearch#chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/common/checkout/scm/git.py&sq=package:chromium&type=cs&l=247.

i.e. change "return self._run_git(['blame', path])" to "return self._run_git(['blame', '--show-email', path])"

The bit I'm unsure about is whether the regexp used to parse the blame lines needs tweaking: https://code.google.com/p/chromium/codesearch#chromium/src/third_party/WebKit/Tools/Scripts/webkitpy/tool/commands/rebaseline.py&q=rebaseline-o-matic&sq=package:chromium&type=cs&l=717

You can run the tool locally by running "webkit-patch rebaseline-o-matic". If it works at all, it should be correct. Otherwise, it will just fail to create a patch, so nothing bad should happen.

[Joel Ong]

Sorry folk, I'll take care of unblocking the tool right now and then take a closer look at fixing the real bug tomorrow.

[Emil A Eklund]

On Mon, Oct 19, 2015 at 6:38 AM, Joel Ong <jo...@chromium.org> wrote:
> Sorry folk, I'll take care of unblocking the tool right now and then take a
> closer look at fixing the real bug tomorrow.

Thanks Joel! I'll stop trying to do manual rebaselines then and give
the tool a chance.

[Walter Korman]

Bot looks like it may need to be kicked/investigated again, I submitted a change this morning that should see a rebaseline and nothing yet. FYI for the group, I am meeting with Joel today to see about helping in a more hands-on manner.

There's nothing but #QUIT# for past day-plus at http://blinkrebaseline.appspot.com/

News at 11! Well, perhaps later this afternoon.

[Joel Ong]

...and it's back in business :)