failing builds

[Lukas Rytz]

I don't know how this could happen, but current master fails the build with

  [partest] testing: [...]/files/run/deeps.scala                                  [FAILED]
  [partest] java.lang.NullPointerException
  [partest] 	at scala.tools.nsc.backend.icode.TypeKinds$class.toTypeKind(TypeKinds.scala:378)
  [partest] 	at scala.tools.nsc.backend.icode.ICodes.toTypeKind(ICodes.scala:20)

Looking at http://scalabuilds.herokuapp.com/ it seems to have started at the following commit:

  https://github.com/scala/scala/commit/8265175ecc42293997d59049f430396c77a2b891

Which was part of pull request https://github.com/scala/scala/pull/1568

My pull request from today (https://github.com/scala/scala/pull/1581) is based on current master

and failed pre-validation with the same error.

Lukas

[Jason Zaugg]

I don't see what the connection could be, but I suggest we revert and monitor the build status.

  https://github.com/scala/scala/pull/1582

-jason

[Lukas Rytz]

it's possible that i was wrong pin-pointing the commit where it started failing,

or that the failiures are spurious.

there are some issues currently with jenkins and the scalabuilds webapp, i'm

looking into it. e.g. lots of failures like this:

  https://scala-webapps.epfl.ch/jenkins/job/scala-checkin/3907/console

[Paul Phillips]

On Tue, Nov 6, 2012 at 5:23 AM, Lukas Rytz <lukas...@epfl.ch> wrote:

it's possible that i was wrong pin-pointing the commit where it started failing,

or that the failiures are spurious.

The failures are not spurious, but they are not deterministic (maybe you intend spurious to mean non-deterministic, but I think of spurious as "inauthentic", like there is nothing wrong with the build.) I ran the tests multiple ways multiple times on https://github.com/scala/scala/pull/1576 and they always all passed, but two different jenkins failed the same way on the same test, which is the same way all those nightlies failed too. I don't know if it's OSX vs. linux somehow, or something else.

[Paul Phillips]

Also, I independently convinced myself the cause was pull request https://github.com/scala/scala/pull/1568 .  I don't see how it could be related either, but it's less far-fetched than any alternative I have.

[Paul Phillips]

And now I've done this twice - convince myself that commit is at fault, only to notice it came into master after I issue #1576 - in other words it isn't even in that pull request, but it failed the same. That would seem more effective at excluding it than indicting it. Back to confused uncertainty for me.

[Lukas Rytz]

On Tue, Nov 6, 2012 at 3:33 PM, Paul Phillips <pa...@improving.org> wrote:

On Tue, Nov 6, 2012 at 5:23 AM, Lukas Rytz <lukas...@epfl.ch> wrote:

it's possible that i was wrong pin-pointing the commit where it started failing,

or that the failiures are spurious.

The failures are not spurious, but they are not deterministic (maybe you intend spurious to mean non-deterministic, but I think of spurious as "inauthentic", like there is nothing wrong with the build.)

yes, that's what i meant by spurious - but you are most likely right, they are just not deterministic

[Paul Phillips]

On Tue, Nov 6, 2012 at 5:23 AM, Lukas Rytz <lukas...@epfl.ch> wrote:

there are some issues currently with jenkins and the scalabuilds webapp, i'm

looking into it. e.g. lots of failures like this:

  https://scala-webapps.epfl.ch/jenkins/job/scala-checkin/3907/console

I'm starting to think what we're really looking at is some kind of tainted build environment. Either it isn't building exactly what it says it is, or something from previous builds infiltrates the current one. Has anyone reproduced the run/deeps.scala failure outside of jenkins? On the same subject, on pull request builds it is very difficult to ascertain what it is that jenkins is testing, which reduces the value of the test. The list of changes given by jenkins is irrelevant because the merge is done from the command line; and one can't reproduce the jenkins repository state by hash because the merge leads to a newly created local hash. I don't have any immediate suggestions how to improve it though...

Oh, confusion lessened somewhat. Retronym's commit wasn't in MY pull request, but JENKINS incorporated it by updating master before the merge and then merging in my merge. In other words I worked against the version of master immediately before retronym's commit, and jenkins worked against the one immediately after.

This decisively says that pull request is the cause, and decisively supports the idea that it is too hard to tell what is being tested. 

[Paul Phillips]

On that note, my immediate suggestion is that the pull request bot be modified to treat merges differently than non-merges. Test merge commits exactly as they are.

[Josh Suereth]

The pull request not is pretty dumb,   it just points the pull request script at a # and it goes.

The script itself is on lukas github project and can be reviewed/fixed as needed.   I'm tinkering in the not today, so let me know what/how it can improve.   Right now it just tells the script to verify PR # for, and no more.  

Jenkins output is terrible, thanks to Jenkins git plugin barfing on job-parameter based merge/branch definitions being unsupported.

If you have suggestions to improve output, I'm all ears.   That bot needs a lot more work to be better.

[Lukas Rytz]

there are some issues currently with jenkins and the scalabuilds webapp, i'm

looking into it.

FYI (Josh, you might be interested) these issues have been resolved now. the

only problem is that jenkins' rest api doesn't show builds in progress anymore,

only completed builds

  https://groups.google.com/d/topic/jenkinsci-users/u88oTJ-nMEU/discussion

that means in the scalabuilds.herokuapp.com webapp will show commits as

either "in build queue" or "build available", but never "build running"

[Josh Suereth]

Ah, so we  only work for older versions of jenkins.  That's pretty fun.

[Lukas Rytz]

For the scalabuilds app it's not a problem, except for what i described below: It will

just think the build is in the queue and hasn't started, and then later find it finished.

Not sure what has to be done for the pr-validator..