buck issues?

112 views
Skip to first unread message

Doug Kelly

unread,
Oct 16, 2015, 1:13:46 AM10/16/15
to Repo and Gerrit Discussion
Following the latest buck update, I've been experiencing weird issues, usually fixed by running builds with --no-cache.

Specifically, the errors (from a completely clean clone):

Compiling module com.google.gerrit.GerritGwtUI

[ERROR] Unexpected internal compiler error

java.lang.NoClassDefFoundError: org/objectweb/asm/MethodVisitor

at java.lang.ClassLoader.defineClass1(Native Method)

at java.lang.ClassLoader.defineClass(ClassLoader.java:760)

at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)

at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)

at java.net.URLClassLoader.access$100(URLClassLoader.java:73)

at java.net.URLClassLoader$1.run(URLClassLoader.java:368)

at java.net.URLClassLoader$1.run(URLClassLoader.java:362)

at java.security.AccessController.doPrivileged(Native Method)

at java.net.URLClassLoader.findClass(URLClassLoader.java:361)

at java.lang.ClassLoader.loadClass(ClassLoader.java:424)

at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)

at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

at java.lang.Class.getDeclaredMethods0(Native Method)

at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)

at java.lang.Class.getDeclaredMethod(Class.java:2128)

at java.io.ObjectStreamClass.getPrivateMethod(ObjectStreamClass.java:1431)

at java.io.ObjectStreamClass.access$1700(ObjectStreamClass.java:72)

at java.io.ObjectStreamClass$2.run(ObjectStreamClass.java:494)

at java.io.ObjectStreamClass$2.run(ObjectStreamClass.java:468)

at java.security.AccessController.doPrivileged(Native Method)

at java.io.ObjectStreamClass.<init>(ObjectStreamClass.java:468)

at java.io.ObjectStreamClass.lookup(ObjectStreamClass.java:365)

at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:602)

at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1623)

at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1518)

at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1774)

at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)

at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2000)

at java.io.ObjectInputStream.defaultReadObject(ObjectInputStream.java:501)

at com.google.gwt.dev.javac.CachedCompilationUnit.readObject(CachedCompilationUnit.java:204)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:497)

at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)

at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1900)

at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)

at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)

at java.io.ObjectInputStream.readArray(ObjectInputStream.java:1707)

at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1345)

at java.io.ObjectInputStream.readObject(ObjectInputStream.java:371)

at com.google.gwt.dev.javac.CompilationUnitArchive.readObject(CompilationUnitArchive.java:124)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:497)

at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)

at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1900)

at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)

at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)

at java.io.ObjectInputStream.readObject(ObjectInputStream.java:371)

at com.google.gwt.dev.javac.CompilationUnitArchive.createFromStream(CompilationUnitArchive.java:59)

at com.google.gwt.dev.javac.CompilationUnitArchive.createFromURL(CompilationUnitArchive.java:66)

at com.google.gwt.dev.ArchivePreloader.preloadArchives(ArchivePreloader.java:62)

at com.google.gwt.dev.Precompile.precompile(Precompile.java:246)

at com.google.gwt.dev.Precompile.precompile(Precompile.java:229)

at com.google.gwt.dev.Precompile.precompile(Precompile.java:145)

at com.google.gwt.dev.Compiler.run(Compiler.java:206)

at com.google.gwt.dev.Compiler.run(Compiler.java:158)

at com.google.gwt.dev.Compiler$1.run(Compiler.java:120)

at com.google.gwt.dev.CompileTaskRunner.doRun(CompileTaskRunner.java:55)

at com.google.gwt.dev.CompileTaskRunner.runWithAppropriateLogger(CompileTaskRunner.java:50)

at com.google.gwt.dev.Compiler.main(Compiler.java:127)

Caused by: java.lang.ClassNotFoundException: org.objectweb.asm.MethodVisitor

at java.net.URLClassLoader.findClass(URLClassLoader.java:381)

at java.lang.ClassLoader.loadClass(ClassLoader.java:424)

at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)

at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

... 63 more


Rolling back to 5bbfb22 seems to work.  After a successful build, it seemed that changes also weren't reflected in the build unless I ran with "--no-cache" again.

--Doug

Luca Milanesio

unread,
Oct 16, 2015, 3:54:15 AM10/16/15
to Doug Kelly, Repo and Gerrit Discussion
Confirmed: the build is broken, see [1].
The CodeMirror upgrade [2] wasn’t so painless apparently.

@DavidO do you see the same issues on your local build?

Luca.
--
--
To unsubscribe, email repo-discuss...@googlegroups.com
More info at http://groups.google.com/group/repo-discuss?hl=en

---
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Luca Milanesio

unread,
Oct 16, 2015, 4:30:30 AM10/16/15
to Edwin Kempin, Repo and Gerrit Discussion
Seems like an compiler (which one? Java or GWT compiler?) internal error:

Compiling module com.google.gerrit.GerritGwtUI
[ERROR] Unexpected internal compiler error
java.lang.NoClassDefFoundError: org/objectweb/asm/MethodVisitor
	at java.lang.ClassLoader.defineClass1(Native Method)

@Edwin: what version of JDK are you using?

Luca.

On 16 Oct 2015, at 09:07, Edwin Kempin <edwin....@gmail.com> wrote:



2015-10-16 9:54 GMT+02:00 Luca Milanesio <luca.mi...@gmail.com>:
Confirmed: the build is broken, see [1].
The CodeMirror upgrade [2] wasn’t so painless apparently.

@DavidO do you see the same issues on your local build?
No Buck issues on my side. The build seems to work fine.

Edwin Kempin

unread,
Oct 16, 2015, 4:33:24 AM10/16/15
to Luca Milanesio, Repo and Gerrit Discussion
2015-10-16 10:30 GMT+02:00 Luca Milanesio <luca.mi...@gmail.com>:
Seems like an compiler (which one? Java or GWT compiler?) internal error:

Compiling module com.google.gerrit.GerritGwtUI
[ERROR] Unexpected internal compiler error
java.lang.NoClassDefFoundError: org/objectweb/asm/MethodVisitor
	at java.lang.ClassLoader.defineClass1(Native Method)

@Edwin: what version of JDK are you using?
Some Google version, I guess that is not helping :-/
 

Luca.

David Pursehouse

unread,
Oct 16, 2015, 4:35:05 AM10/16/15
to Edwin Kempin, Luca Milanesio, Repo and Gerrit Discussion
On 10/16/2015 05:33 PM, Edwin Kempin wrote:
>
>
> 2015-10-16 10:30 GMT+02:00 Luca Milanesio <luca.mi...@gmail.com
> <mailto:luca.mi...@gmail.com>>:
>
> Seems like an compiler (which one? Java or GWT compiler?) internal
> error:
>
> Compiling module com.google.gerrit.GerritGwtUI
> [ERROR] Unexpected internal compiler error
> java.lang.NoClassDefFoundError: org/objectweb/asm/MethodVisitor
> at java.lang.ClassLoader.defineClass1(Native Method)
>
>
> @Edwin: what version of JDK are you using?
>
> Some Google version, I guess that is not helping :-/
>

It's working OK for me:

$ java -version
java version "1.7.0_79"
OpenJDK Runtime Environment (IcedTea 2.5.6) (7u79-2.5.6-0ubuntu1.12.04.1)
OpenJDK 64-Bit Server VM (build 24.79-b02, mixed mode)


Matthias Sohn

unread,
Oct 16, 2015, 5:33:13 AM10/16/15
to David Pursehouse, Edwin Kempin, Luca Milanesio, Repo and Gerrit Discussion
works ok here on Oracle JVM

gerrit (master)]$ java -version
java version "1.7.0_80"
Java(TM) SE Runtime Environment (build 1.7.0_80-b15)
Java HotSpot(TM) 64-Bit Server VM (build 24.80-b11, mixed mode) 

-Matthias

David Ostrovsky

unread,
Oct 16, 2015, 6:36:43 AM10/16/15
to Repo and Gerrit Discussion, doug...@gmail.com

Am Freitag, 16. Oktober 2015 09:54:15 UTC+2 schrieb lucamilanesio:
Confirmed: the build is broken, see [1].
The CodeMirror upgrade [2] wasn’t so painless apparently.

@DavidO do you see the same issues on your local build?

Nope. Doug mentioned, that problem is fixed by cache invalidation,
so I wonder, if buck clean, or switch to older Buck versions reliably
triggers the problem? It would also help if someone could bisect it.

Doug Kelly

unread,
Oct 16, 2015, 7:38:59 AM10/16/15
to Repo and Gerrit Discussion, doug...@gmail.com
For the record, my JVM:
java version "1.8.0_60"
Java(TM) SE Runtime Environment (build 1.8.0_60-b27)
Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode)

I also want to add, the build just *appears* broken.  But, once you get a working build by invalidating the cache, additional changes don't seem to reliably compile in without invalidating the cache, too (it'll appear to compile, then not work -- like your change wasn't there).  At best, I think build results are unreliable right now. 

David Ostrovsky

unread,
Oct 16, 2015, 8:11:43 AM10/16/15
to Repo and Gerrit Discussion, doug...@gmail.com

Am Freitag, 16. Oktober 2015 13:38:59 UTC+2 schrieb Doug Kelly:
On Friday, October 16, 2015 at 5:36:43 AM UTC-5, David Ostrovsky wrote:

Am Freitag, 16. Oktober 2015 09:54:15 UTC+2 schrieb lucamilanesio:
Confirmed: the build is broken, see [1].
The CodeMirror upgrade [2] wasn’t so painless apparently.

@DavidO do you see the same issues on your local build?

Nope. Doug mentioned, that problem is fixed by cache invalidation,
so I wonder, if buck clean, or switch to older Buck versions reliably
triggers the problem? It would also help if someone could bisect it.

For the record, my JVM:
java version "1.8.0_60"
Java(TM) SE Runtime Environment (build 1.8.0_60-b27)
Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode)

I also want to add, the build just *appears* broken.

OK, got it! Has nothing to do with Java compiler.

Looks like the recently changed bottom-up to top-down dependency graph-traversing problem,
related to gwt_binray() rule.

The steps to reproduce on most recent master:

1. invalidate cache
2. buck build gerrit
3. buck clean
4. buck build gerrit
5. change anything related to GWT, say edit this file gerrit-gwtui/src/main/java/com/google/gerrit/client/Gerrit.java
6. buck build gerrit

That's because: asm libraries weren't fetched from the cache:

  $ buck targets --show_output //lib/ow2:ow2-asm
  Using buckd.
  //lib/ow2:ow2-asm buck-out/gen/lib/ow2/ow2-asm.jar

but

  $ ls -all buck-out/gen/lib/ow2/ow2-asm.jar
  ls: cannot access buck-out/gen/lib/ow2/ow2-asm.jar: No such file or directory

Workaround for now enforce bottom-up dependency graph traversing:

  $ buck build --deep gerrit

fixed it.

Investigating whether this is a issue in our toolchain or Buck bug...

Doug Kelly

unread,
Oct 16, 2015, 8:17:16 AM10/16/15
to Repo and Gerrit Discussion, doug...@gmail.com
Nice!  The problem that concerns me is, I made a non-GWT change, and while it didn't crash in the compile, it didn't reflect my change, either.  I was testing

David Ostrovsky

unread,
Oct 16, 2015, 8:32:39 AM10/16/15
to Repo and Gerrit Discussion, doug...@gmail.com
Sounds like a different issue to me. Disable Buck daemon and/or watchman?

Luca Milanesio

unread,
Oct 16, 2015, 9:02:24 AM10/16/15
to David Ostrovsky, Repo and Gerrit Discussion, doug...@gmail.com
Wow, interesting ;-)

Let me try the “—deep” on ci-gerrit …

Luca.

David Ostrovsky

unread,
Oct 16, 2015, 9:21:25 AM10/16/15
to Repo and Gerrit Discussion, doug...@gmail.com

Am Freitag, 16. Oktober 2015 15:02:24 UTC+2 schrieb lucamilanesio:
Wow, interesting ;-)

Let me try the “—deep” on ci-gerrit …

Doug Kelly

unread,
Oct 16, 2015, 12:09:10 PM10/16/15
to Repo and Gerrit Discussion, doug...@gmail.com
I think you're right, but disabling buckd/watchman doesn't seem to help.  I think the only "fix" is disabling the cache... but it's something specific about that one change, since I've not noticed this on anything else?  I'm at least concerned that I've got a case where builds are inaccurate... that seems very troubling from a reliability standpoint. (And, the issue seems to be reproducible on my machine, even with clean clones, buck clean, etc.)

David Ostrovsky

unread,
Oct 16, 2015, 1:58:47 PM10/16/15
to Repo and Gerrit Discussion, doug...@gmail.com
Can you provide exact steps how to reproduce inaccurate build results?
The only Buck problem that I'm aware of is related to symbolic links: [1].
Other than that it should just work. I suspect problem in your setup, unless
you convince me otherwise ;-)


Doug Kelly

unread,
Oct 16, 2015, 2:38:38 PM10/16/15
to Repo and Gerrit Discussion, doug...@gmail.com
That's fair.  I'd be suspicious, too, if it weren't a completely clean setup which I've done absolutely nothing strange on.  But, testing this on another laptop entirely (that I've been using with Gerrit builds before)... here are the steps I followed:
1) git pull --rebase origin/master
2) buck build gerrit (new version of buck builds)
3) java -jar gerrit-2.10.7.war init -d ../test_gerrit-upgrade (download bouncy castle)
4) java -jar buck-out/gen/gerrit/gerrit.war init -d ../test_gerrit-upgrade (see that bouncy castle renames and downloads... and the messages have a line break between)
5) git revert a11d9c2dfa40f1894092a05cc31d18108b94a8d0 (this fixed the messages being run together on the same line)
6) buck build gerrit
7) java -jar gerrit-2.10.7.war init -d ../test_gerrit-upgrade2 (download bouncy castle)
8) java -jar buck-out/gen/gerrit/gerrit.war init -d ../test_gerrit-upgrade2 (see that the line break is still present between the rename/download)

This seems to be 100% reproducible on two systems... so it seems that somehow, buck isn't resolving the change to gerrit-pgm/src/main/java/com/google/gerrit/pgm/init/LibraryDownloader.java.  Running buck build gerit --no-cache doesn't seem to help, either.  Even adding NO_BUCKD=1 to my environment doesn't seem to make a change.  Removing the buck cache does fix the issue.  This is about all I can think of to make sure environment is not in the loop, as much as possible.

Any other thoughts?  Still want to blame my environment(s)? :)

David Ostrovsky

unread,
Oct 16, 2015, 6:55:42 PM10/16/15
to Repo and Gerrit Discussion, doug...@gmail.com

Bruce Zu

unread,
Feb 26, 2016, 8:05:57 PM2/26/16
to Repo and Gerrit Discussion, doug...@gmail.com
Hi Luca,
how about add below line into  GerritForge CI

buck targets | xargs buck build && ./tools/eclipse/project.py && ./tools/eclipse/project.py --src


Thus let GerritForge CI cover more detail test for new commit.

Thanks!

Bruce

David Pursehouse

unread,
Mar 1, 2016, 8:41:00 PM3/1/16
to Bruce Zu, Repo and Gerrit Discussion, doug...@gmail.com
On Sat, Feb 27, 2016 at 10:06 AM Bruce Zu <zu.bruc...@gmail.com> wrote:
Hi Luca,
how about add below line into  GerritForge CI

buck targets | xargs buck build && ./tools/eclipse/project.py && ./tools/eclipse/project.py --src


It's not necessary to explicitly call project.py twice.

Since [1] (merged on master) the --src option is deprecated and the script will always download sources by default even without the option.

It should be enough to just call "project.py --src".  This will have the same result on all branches.



 
--

Bruce Zu

unread,
Mar 1, 2016, 8:50:45 PM3/1/16
to Repo and Gerrit Discussion, zu.bruc...@gmail.com, doug...@gmail.com


On Tuesday, March 1, 2016 at 5:41:00 PM UTC-8, David Pursehouse wrote:
On Sat, Feb 27, 2016 at 10:06 AM Bruce Zu <zu.bruc...@gmail.com> wrote:
Hi Luca,
how about add below line into  GerritForge CI

buck targets | xargs buck build && ./tools/eclipse/project.py && ./tools/eclipse/project.py --src


It's not necessary to explicitly call project.py twice.

Since [1] (merged on master) the --src option is deprecated and the script will always download sources by default even without the option.

It should be enough to just call "project.py --src".  This will have the same result on all branches.

smart idea!
Reply all
Reply to author
Forward
0 new messages