buck issues?

[Doug Kelly]

Following the latest buck update, I've been experiencing weird issues, usually fixed by running builds with --no-cache.

Specifically, the errors (from a completely clean clone):

Compiling module com.google.gerrit.GerritGwtUI

[ERROR] Unexpected internal compiler error

java.lang.NoClassDefFoundError: org/objectweb/asm/MethodVisitor

at java.lang.ClassLoader.defineClass1(Native Method)

at java.lang.ClassLoader.defineClass(ClassLoader.java:760)

at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)

at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)

at java.net.URLClassLoader.access$100(URLClassLoader.java:73)

at java.net.URLClassLoader$1.run(URLClassLoader.java:368)

at java.net.URLClassLoader$1.run(URLClassLoader.java:362)

at java.security.AccessController.doPrivileged(Native Method)

at java.net.URLClassLoader.findClass(URLClassLoader.java:361)

at java.lang.ClassLoader.loadClass(ClassLoader.java:424)

at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)

at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

at java.lang.Class.getDeclaredMethods0(Native Method)

at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)

at java.lang.Class.getDeclaredMethod(Class.java:2128)

at java.io.ObjectStreamClass.getPrivateMethod(ObjectStreamClass.java:1431)

at java.io.ObjectStreamClass.access$1700(ObjectStreamClass.java:72)

at java.io.ObjectStreamClass$2.run(ObjectStreamClass.java:494)

at java.io.ObjectStreamClass$2.run(ObjectStreamClass.java:468)

at java.security.AccessController.doPrivileged(Native Method)

at java.io.ObjectStreamClass.<init>(ObjectStreamClass.java:468)

at java.io.ObjectStreamClass.lookup(ObjectStreamClass.java:365)

at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:602)

at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1623)

at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1518)

at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1774)

at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)

at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2000)

at java.io.ObjectInputStream.defaultReadObject(ObjectInputStream.java:501)

at com.google.gwt.dev.javac.CachedCompilationUnit.readObject(CachedCompilationUnit.java:204)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:497)

at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)

at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1900)

at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)

at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)

at java.io.ObjectInputStream.readArray(ObjectInputStream.java:1707)

at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1345)

at java.io.ObjectInputStream.readObject(ObjectInputStream.java:371)

at com.google.gwt.dev.javac.CompilationUnitArchive.readObject(CompilationUnitArchive.java:124)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:497)

at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)

at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1900)

at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)

at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)

at java.io.ObjectInputStream.readObject(ObjectInputStream.java:371)

at com.google.gwt.dev.javac.CompilationUnitArchive.createFromStream(CompilationUnitArchive.java:59)

at com.google.gwt.dev.javac.CompilationUnitArchive.createFromURL(CompilationUnitArchive.java:66)

at com.google.gwt.dev.ArchivePreloader.preloadArchives(ArchivePreloader.java:62)

at com.google.gwt.dev.Precompile.precompile(Precompile.java:246)

at com.google.gwt.dev.Precompile.precompile(Precompile.java:229)

at com.google.gwt.dev.Precompile.precompile(Precompile.java:145)

at com.google.gwt.dev.Compiler.run(Compiler.java:206)

at com.google.gwt.dev.Compiler.run(Compiler.java:158)

at com.google.gwt.dev.Compiler$1.run(Compiler.java:120)

at com.google.gwt.dev.CompileTaskRunner.doRun(CompileTaskRunner.java:55)

at com.google.gwt.dev.CompileTaskRunner.runWithAppropriateLogger(CompileTaskRunner.java:50)

at com.google.gwt.dev.Compiler.main(Compiler.java:127)

Caused by: java.lang.ClassNotFoundException: org.objectweb.asm.MethodVisitor

at java.net.URLClassLoader.findClass(URLClassLoader.java:381)

at java.lang.ClassLoader.loadClass(ClassLoader.java:424)

at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)

at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

... 63 more

Rolling back to 5bbfb22 seems to work.  After a successful build, it seemed that changes also weren't reflected in the build unless I ran with "--no-cache" again.

--Doug

[Luca Milanesio]

Confirmed: the build is broken, see [1].

The CodeMirror upgrade [2] wasn’t so painless apparently.

@DavidO do you see the same issues on your local build?

Luca.

[1] https://gerrit-ci.gerritforge.com/job/Gerrit-master/451/console

[2] https://gerrit-review.googlesource.com/#/c/71600/

--
--
To unsubscribe, email repo-discuss...@googlegroups.com
More info at http://groups.google.com/group/repo-discuss?hl=en

---
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Luca Milanesio]

Seems like an compiler (which one? Java or GWT compiler?) internal error:

Compiling module com.google.gerrit.GerritGwtUI
[ERROR] Unexpected internal compiler error
java.lang.NoClassDefFoundError: org/objectweb/asm/MethodVisitor
	at java.lang.ClassLoader.defineClass1(Native Method)

@Edwin: what version of JDK are you using?

Luca.

On 16 Oct 2015, at 09:07, Edwin Kempin <edwin....@gmail.com> wrote:

2015-10-16 9:54 GMT+02:00 Luca Milanesio <luca.mi...@gmail.com>:

Confirmed: the build is broken, see [1].

The CodeMirror upgrade [2] wasn’t so painless apparently.

@DavidO do you see the same issues on your local build?

No Buck issues on my side. The build seems to work fine.

[Edwin Kempin]

2015-10-16 10:30 GMT+02:00 Luca Milanesio <luca.mi...@gmail.com>:

Seems like an compiler (which one? Java or GWT compiler?) internal error:

Compiling module com.google.gerrit.GerritGwtUI
[ERROR] Unexpected internal compiler error
java.lang.NoClassDefFoundError: org/objectweb/asm/MethodVisitor
	at java.lang.ClassLoader.defineClass1(Native Method)

@Edwin: what version of JDK are you using?

Some Google version, I guess that is not helping :-/

 

Luca.

[David Pursehouse]

On 10/16/2015 05:33 PM, Edwin Kempin wrote:
>
>
> 2015-10-16 10:30 GMT+02:00 Luca Milanesio <luca.mi...@gmail.com

> <mailto:luca.mi...@gmail.com>>:

>
> Seems like an compiler (which one? Java or GWT compiler?) internal
> error:
>
> Compiling module com.google.gerrit.GerritGwtUI
> [ERROR] Unexpected internal compiler error
> java.lang.NoClassDefFoundError: org/objectweb/asm/MethodVisitor
> at java.lang.ClassLoader.defineClass1(Native Method)
>
>
> @Edwin: what version of JDK are you using?
>
> Some Google version, I guess that is not helping :-/
>

It's working OK for me:

$ java -version
java version "1.7.0_79"
OpenJDK Runtime Environment (IcedTea 2.5.6) (7u79-2.5.6-0ubuntu1.12.04.1)
OpenJDK 64-Bit Server VM (build 24.79-b02, mixed mode)

[Matthias Sohn]

works ok here on Oracle JVM

gerrit (master)]$ java -version

java version "1.7.0_80"

Java(TM) SE Runtime Environment (build 1.7.0_80-b15)

Java HotSpot(TM) 64-Bit Server VM (build 24.80-b11, mixed mode) 

-Matthias

[David Ostrovsky]

Am Freitag, 16. Oktober 2015 09:54:15 UTC+2 schrieb lucamilanesio:

Confirmed: the build is broken, see [1].

The CodeMirror upgrade [2] wasn’t so painless apparently.

@DavidO do you see the same issues on your local build?

Nope. Doug mentioned, that problem is fixed by cache invalidation,

so I wonder, if buck clean, or switch to older Buck versions reliably

triggers the problem? It would also help if someone could bisect it.

[Doug Kelly]

For the record, my JVM:

java version "1.8.0_60"

Java(TM) SE Runtime Environment (build 1.8.0_60-b27)

Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode)

I also want to add, the build just *appears* broken.  But, once you get a working build by invalidating the cache, additional changes don't seem to reliably compile in without invalidating the cache, too (it'll appear to compile, then not work -- like your change wasn't there).  At best, I think build results are unreliable right now. 

[David Ostrovsky]

Am Freitag, 16. Oktober 2015 13:38:59 UTC+2 schrieb Doug Kelly:

On Friday, October 16, 2015 at 5:36:43 AM UTC-5, David Ostrovsky wrote:

Am Freitag, 16. Oktober 2015 09:54:15 UTC+2 schrieb lucamilanesio:

Confirmed: the build is broken, see [1].

The CodeMirror upgrade [2] wasn’t so painless apparently.

@DavidO do you see the same issues on your local build?

Nope. Doug mentioned, that problem is fixed by cache invalidation,

so I wonder, if buck clean, or switch to older Buck versions reliably

triggers the problem? It would also help if someone could bisect it.

For the record, my JVM:

java version "1.8.0_60"

Java(TM) SE Runtime Environment (build 1.8.0_60-b27)

Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode)

I also want to add, the build just *appears* broken.

OK, got it! Has nothing to do with Java compiler.

Looks like the recently changed bottom-up to top-down dependency graph-traversing problem,

related to gwt_binray() rule.

The steps to reproduce on most recent master:

1. invalidate cache

2. buck build gerrit

3. buck clean

4. buck build gerrit

5. change anything related to GWT, say edit this file gerrit-gwtui/src/main/java/com/google/gerrit/client/Gerrit.java

6. buck build gerrit

7. crash: http://paste.openstack.org/show/476475

That's because: asm libraries weren't fetched from the cache:

  $ buck targets --show_output //lib/ow2:ow2-asm

  Using buckd.

  //lib/ow2:ow2-asm buck-out/gen/lib/ow2/ow2-asm.jar

but

  $ ls -all buck-out/gen/lib/ow2/ow2-asm.jar

  ls: cannot access buck-out/gen/lib/ow2/ow2-asm.jar: No such file or directory

Workaround for now enforce bottom-up dependency graph traversing:

  $ buck build --deep gerrit

fixed it.

Investigating whether this is a issue in our toolchain or Buck bug...

[Doug Kelly]

Nice!  The problem that concerns me is, I made a non-GWT change, and while it didn't crash in the compile, it didn't reflect my change, either.  I was testing

https://gerrit-review.googlesource.com/71530/ .... 

[David Ostrovsky]

Sounds like a different issue to me. Disable Buck daemon and/or watchman?

[Luca Milanesio]

Wow, interesting ;-)

Let me try the “—deep” on ci-gerrit …

Luca.

[David Ostrovsky]

Am Freitag, 16. Oktober 2015 15:02:24 UTC+2 schrieb lucamilanesio:

Wow, interesting ;-)

Let me try the “—deep” on ci-gerrit …

Fixed in: [1]. 

* [1] https://gerrit-review.googlesource.com/71623

[Doug Kelly]

I think you're right, but disabling buckd/watchman doesn't seem to help.  I think the only "fix" is disabling the cache... but it's something specific about that one change, since I've not noticed this on anything else?  I'm at least concerned that I've got a case where builds are inaccurate... that seems very troubling from a reliability standpoint. (And, the issue seems to be reproducible on my machine, even with clean clones, buck clean, etc.)

[David Ostrovsky]

Can you provide exact steps how to reproduce inaccurate build results?

The only Buck problem that I'm aware of is related to symbolic links: [1].

Other than that it should just work. I suspect problem in your setup, unless

you convince me otherwise ;-)

* [1] https://github.com/facebook/buck/issues/341

[Doug Kelly]

That's fair.  I'd be suspicious, too, if it weren't a completely clean setup which I've done absolutely nothing strange on.  But, testing this on another laptop entirely (that I've been using with Gerrit builds before)... here are the steps I followed:

1) git pull --rebase origin/master

2) buck build gerrit (new version of buck builds)

3) java -jar gerrit-2.10.7.war init -d ../test_gerrit-upgrade (download bouncy castle)

4) java -jar buck-out/gen/gerrit/gerrit.war init -d ../test_gerrit-upgrade (see that bouncy castle renames and downloads... and the messages have a line break between)

5) git revert a11d9c2dfa40f1894092a05cc31d18108b94a8d0 (this fixed the messages being run together on the same line)

6) buck build gerrit

7) java -jar gerrit-2.10.7.war init -d ../test_gerrit-upgrade2 (download bouncy castle)

8) java -jar buck-out/gen/gerrit/gerrit.war init -d ../test_gerrit-upgrade2 (see that the line break is still present between the rename/download)

This seems to be 100% reproducible on two systems... so it seems that somehow, buck isn't resolving the change to gerrit-pgm/src/main/java/com/google/gerrit/pgm/init/LibraryDownloader.java.  Running buck build gerit --no-cache doesn't seem to help, either.  Even adding NO_BUCKD=1 to my environment doesn't seem to make a change.  Removing the buck cache does fix the issue.  This is about all I can think of to make sure environment is not in the loop, as much as possible.

Any other thoughts?  Still want to blame my environment(s)? :)

[David Ostrovsky]

Fixed in: [1].

* [1] https://gerrit-review.googlesource.com/71650

[Bruce Zu]

Hi Luca,

how about add below line into  GerritForge CI

buck targets | xargs buck build && ./tools/eclipse/project.py && ./tools/eclipse/project.py --src

Thus let GerritForge CI cover more detail test for new commit.

Thanks!

Bruce

[David Pursehouse]

On Sat, Feb 27, 2016 at 10:06 AM Bruce Zu <zu.bruc...@gmail.com> wrote:

Hi Luca,

how about add below line into  GerritForge CI

buck targets | xargs buck build && ./tools/eclipse/project.py && ./tools/eclipse/project.py --src

It's not necessary to explicitly call project.py twice.

Since [1] (merged on master) the --src option is deprecated and the script will always download sources by default even without the option.

It should be enough to just call "project.py --src".  This will have the same result on all branches.

[1] https://gerrit-review.googlesource.com/#/c/75130/

 

--

[Bruce Zu]

On Tuesday, March 1, 2016 at 5:41:00 PM UTC-8, David Pursehouse wrote:

On Sat, Feb 27, 2016 at 10:06 AM Bruce Zu <zu.bruc...@gmail.com> wrote:

Hi Luca,

how about add below line into  GerritForge CI

buck targets | xargs buck build && ./tools/eclipse/project.py && ./tools/eclipse/project.py --src

It's not necessary to explicitly call project.py twice.

Since [1] (merged on master) the --src option is deprecated and the script will always download sources by default even without the option.

It should be enough to just call "project.py --src".  This will have the same result on all branches.

[1] https://gerrit-review.googlesource.com/#/c/75130/

smart idea！