Weaving Kilim Tasks

[ChromoX]

I was wondering if anyone has a build script that will build a
directory of classes with kilim and then weave them.

I have tried using the command line, but I end up having to change the
class path and whenever I do something always gets excluded...

Examples:

java kilim.tools.Weaver -d ../classes/ kilim.examples.SimpleTask
Error weaving null
java.lang.NoClassDefFoundError: org/objectweb/asm/ClassVisitor
at kilim.tools.Weaver.weaveClass(Weaver.java:114)
at kilim.tools.Weaver.main(Weaver.java:71)
Caused by: java.lang.ClassNotFoundException:
org.objectweb.asm.ClassVisitor
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)

java -cp ../libs/:CLASSPATH kilim.tools.Weaver -d ../classes/
kilim.examples.SimpleTask
Exception in thread "main" java.lang.NoClassDefFoundError: kilim/tools/
Weaver
Caused by: java.lang.ClassNotFoundException: kilim.tools.Weaver
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)

java -cp ../libs/:./:CLASSPATH kilim.tools.Weaver -d ../classes/
kilim.examples.SimpleTask
Error weaving null
java.lang.NoClassDefFoundError: org/objectweb/asm/ClassVisitor
at kilim.tools.Weaver.weaveClass(Weaver.java:114)
at kilim.tools.Weaver.main(Weaver.java:71)
Caused by: java.lang.ClassNotFoundException:
org.objectweb.asm.ClassVisitor
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)

[Sriram Srinivasan]

The clue is in the error.

> java.lang.NoClassDefFoundError: org/objectweb/asm/ClassVisitor

It needs the asm library. Put kilim/libs/asm-all-2.2.3.jar in the
classpath.

[ChromoX]

When I put it in the classpath it loses the current path in the class
path...

java -cp ../libs/:CLASSPATH kilim.tools.Weaver -d ../classes/
kilim.examples.SimpleTask
Exception in thread "main" java.lang.NoClassDefFoundError: kilim/
tools/
Weaver
Caused by: java.lang.ClassNotFoundException: kilim.tools.Weaver
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:
301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)

And then if I add my current path and the libs directory to the
classpath

java -cp ../libs/:./:CLASSPATH kilim.tools.Weaver -d ../classes/
kilim.examples.SimpleTask
Error weaving null
java.lang.NoClassDefFoundError: org/objectweb/asm/ClassVisitor
at kilim.tools.Weaver.weaveClass(Weaver.java:114)
at kilim.tools.Weaver.main(Weaver.java:71)
Caused by: java.lang.ClassNotFoundException:
org.objectweb.asm.ClassVisitor
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:
301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)

[Jason Pell]

Question any thought about updating to use asm 3.3?

Sent from my iPhone

[Sriram Srinivasan]

On Nov 7, 2010, at 11:13 PM, ChromoX wrote:

> When I put it in the classpath it loses the current path in the class
> path...
>
> java -cp ../libs/:CLASSPATH kilim.tools.Weaver -d ../classes/
> kilim.examples.SimpleTask

You want to have the jar in the classpath.

java -cp ../libs/asm-all-2.2.3.jar ....

See the file test.sh.

[Sriram Srinivasan]

On Nov 8, 2010, at 12:38 AM, Jason Pell wrote:

> Question any thought about updating to use asm 3.3?

Any reason to update it, other than keeping with the times? asm3.3
adds a lot of code, but nothing that changes what I need from it.

--sriram.

[Jason Pell]

I am using kilim with CXF and CXF 2.3.0 added a dependency on ASM 3.3!
I have managed to exclude 3.3 and
specify 2.2.3 as dependency and that is supported. It seems like
there are a few api changes
that would have to be updated to bring it up to 3.3.

but was just curious.

[Sriram Srinivasan]

I see. I was loathe to make the changes just to keep upgrading, but I
would upgrade the priority if it becomes a showstopper.

[xiode04]

Is there any easy weaving integration with NetBeans? Say I had a
project in Netbeans at the directory:

~/Documents/program_name/project_name
---------------
> lib/
---------------
> src/
---------------
> nbproject/
---------------
> build.xml

The build-file that NetBeans creates is enormous (several hundred
lines) and I'm just looking for an easy way to weave all of the
classes in src/ including all of the sub-folders. Maybe I'm retarded,
but I can't get it to work =\ Currently the contents of lib/ are just
kilim.jar

[xiode04]

Sorry, see below. It raped my directory formatting >.<

[xiode04]

*sigh*
One last time, the good 'ol fashion way:

cd ~/Documents/program_name/project_name
ls
lib/ src/ nbproject/ build.xml

cd lib
ls
kalim.jar

cd ../src/
ls
Class1.java Class2.java Class3.java /subfolder1 /subfolder2


That's what I'm working with ^ ^

[Jason Pell]

For now it's working fine and CXF developers are happy enough that 2.2.3 is still supported so no biggie

Sent from my iPhone

[ChromoX]

I am again trying to weave everything together and these are the steps
I am using currently...

Three components:
Kilim
My Project Library
My Project Application

1: Build and Weave kilim; package into a jar
2. Add kilim.jar to my classpath, compile and weave my library
3. Add my project library to my classpath, and compile my application,
then after compile when I try and weave I get this error

java.lang.NoClassDefFoundError: API/Ba
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:169)
at
kilim.mirrors.RuntimeClassMirrors.classForName(RuntimeClassMirrors.java:
154)
at kilim.analysis.Detector.getPausableStatus(Detector.java:64)
at kilim.analysis.MethodFlow.visitMethodInsn(MethodFlow.java:202)
at org.objectweb.asm.ClassReader.accept(Unknown Source)
at org.objectweb.asm.ClassReader.accept(Unknown Source)
at kilim.analysis.ClassFlow.analyze(ClassFlow.java:81)
at kilim.analysis.ClassWeaver.weave(ClassWeaver.java:50)
at kilim.analysis.ClassWeaver.<init>(ClassWeaver.java:41)
at kilim.tools.Weaver.weaveFile(Weaver.java:96)
at kilim.tools.Weaver.main(Weaver.java:67)
Caused by: java.lang.ClassNotFoundException: API.Ba

at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)

... 12 more

Its trying to find OAPI.Bar and this only happens during weaving.

Any ideas?

[Sriram Srinivasan]

Put the weaver's output directory in the classpath.

[ChromoX]

export CLASSPATH=./classes:./lib/asm-all-2.2.3.jar:./lib/kilim.jar:./
lib/OAPI.jar:$CLASSPATH

rm -rf ./classes
mkdir ./classes

echo Compiling Project
javac -g -d ./classes `find . -name "*.java" `

echo Weaving....
# Weave all files under ./classes, compiling the tests to
java -ea kilim.tools.Weaver -d ./classes -x "ExInvalid|test" ./classes


Same error after running that script

[ChromoX]

I tried another way....

I compiled each project including kilim independently, then wove each
of them, and tried to run it like so:

java -cp ./kilim:./ cayman.Main
Exception in thread "main" java.lang.NoClassDefFoundError: kilim/S_O4
Caused by: java.lang.ClassNotFoundException: kilim.S_O4

at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)

[xiode04]

ChromoX and I are collaborating on the same project.

Given a library project directory, an application project directory,
and Kilim obviously, I'm currently trying:

1) Compile the library project and weave it (successful). I then JAR
it and move it to /lib/ in our application project.

Now, in our application project, I extract all of the woven class
files from kilim.jar and our library.jar to ./classes/
2) Compile the application project and output it to ./classes/
(successful).
3) Then, I call the weaver on all of ./classes/ in the application
project's directory (successful).

Now in ./classes/ in the application directory, we have three
distinct, woven folders (kilim, application, and library).
As you might assume. Main is a class that resides in the application
folder.

However, no matter what way I seem to call it, I can't get Main to
run.
ex: java application.Main
java -cp ../lib/kilim.jar:../lib/library.jar application.Main

A typical error:
Exception in thread "main" java.lang.NoClassDefFoundError: API/Ba
Caused by: java.lang.ClassNotFoundException: API.Ba
at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
Could not find the main class: application.Main. Program will exit.

The real class it's looking for is OAPI.Bar (where OAPI is the library
I've been referencing above). As you can see, it's truncating the
first and last letters of the class name/path.


Is there some way I should reorganize my project or an alternative way
to call it from the command-line? All of the weaving seems
successful, but for whatever reason we can't get it to run.

[Sriram Srinivasan]

> 3) Then, I call the weaver on all of ./classes/ in the application
> project's directory (successful).
>
> Now in ./classes/ in the application directory, we have three
> distinct, woven folders (kilim, application, and library).
> As you might assume. Main is a class that resides in the application
> folder.
>
> However, no matter what way I seem to call it, I can't get Main to
> run.
> ex: java application.Main
> java -cp ../lib/kilim.jar:../lib/library.jar application.Main
>

Shouldn't ./classes be in the classpath?

> A typical error:
> Exception in thread "main" java.lang.NoClassDefFoundError: API/Ba

> The real class it's looking for is OAPI.Bar (where OAPI is the library
> I've been referencing above). As you can see, it's truncating the
> first and last letters of the class name/path.

This is strange.

I can't imagine this to be a weaver error. But in case we run out of
more credible explanations, you can see which classes are being loaded
this way:
java -ea kilim.tools.DumpClass | perl -ne 'print $1, "\n" if (/(L[^ ]
+;)/);' | sort | uniq

>
>
> Is there some way I should reorganize my project or an alternative way
> to call it from the command-line? All of the weaving seems
> successful, but for whatever reason we can't get it to run.
>

No, there should be no special arrangement required.

--sriram.

[Sriram Srinivasan]

> I can't imagine this to be a weaver error. But in case we run out of
> more credible explanations, you can see which classes are being
> loaded this way:
> java -ea kilim.tools.DumpClass | perl -ne 'print $1, "\n" if (/(L[^ ]
> +;)/);' | sort | uniq
>

oops. I forgot the argument to DumpClass. You can also egrep. I just
use perl reflexively.

java -ea kilim.tools.DumpClass classes/*.class | egrep -o 'L[^ ]+;' |
sort | uniq

See if the names have been mangled.

[xiode04]

I tried making a run.sh script that consisted of (called from the root
of my application directory):

echo "Trying to run Main"
java -cp ./classes:./lib/asm-all-2.2.3.jar:./lib/kilim.jar:./lib/
mylibrary.jar application.Main

Same exact error as before (before I was calling that "java" command
from within ./classes/ which I forgot to include). Is there a better
way to weave multiple projects that you would recommend?

[Sriram Srinivasan]

Have you marked main() to be Pausable?

If so, don't !

No, not really. If there are multiple projects, there are multiple
output directories, so you would need to handle them independently
anyway.

[xiode04]

Ok, progress. You were right in assuming that public static void
main() was marked Pausable, and that was because I had to do that in
order to get rid of some pesky errors (main calls Pausable methods).
I commented it out for the meantime, and using my run.sh script from
above, I can get Main to run (it's an interactive prompt that calls
other classes and performs functions).

However, while it works to a certain degree, anytime I use this
interactive prompt to do something that references a class in the
library project, it craps out with an error similar to last time:

Exception in thread "main" java.lang.NoClassDefFoundError: API/Ba

at application.Main.main(Main.java:60)
Caused by: java.lang.ClassNotFoundException: API.Ba
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)

at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)

... 1 more


Just to reiterate, the call I'm making via the terminal is:

java -cp ./classes:./lib/asm-all-2.2.3.jar:./lib/kilim.jar:./lib/

library.jar application.Main

./classes/ has all of the woven .class files and is structured like:
------- kilim/
------- library/
------- application/ (which contains Main)


Any ideas?

[Sriram Srinivasan]

On Nov 14, 2010, at 11:34 PM, xiode04 wrote:

> Ok, progress. You were right in assuming that public static void
> main() was marked Pausable, and that was because I had to do that in
> order to get rid of some pesky errors (main calls Pausable methods).

Well, since Pausable is a semantic construct, and main() doesn't
follow the same semantics. it is a bit more than a pesky compilation
error.

The whole idea of Pausable is that a method can _prematurely_ return
to its caller, and have the caller do the same. A non pausable method
such as main() is not geared to be told by its callee that it isn't
done yet: the implicit contract is that if the method returns, it has
finished its job.

To avoid this, call your pausable methods from within a task started
from main. main() can exit, but the program will not quit.
Alternatively, if you want to know when the task is finished, and have
main do other things subsequently, you can call join():

main()
Task t = new Mytask().start();
t.joinb(); // blocks thread until task t finish.
}

>
> However, while it works to a certain degree, anytime I use this
> interactive prompt to do something that references a class in the
> library project, it craps out with an error similar to last time:
>
> Exception in thread "main" java.lang.NoClassDefFoundError: API/Ba
>

.....

>
> Just to reiterate, the call I'm making via the terminal is:
> java -cp ./classes:./lib/asm-all-2.2.3.jar:./lib/kilim.jar:./lib/
> library.jar application.Main
>

Did you try the DumpClass one-liner I sent you? I'd like to know if
there is something like API/Ba referenced in the class files.

It'd be better if you could replicate this error with a much much
smaller example that mimics your project structure.

[xiode04]

From the root application directory, calling:

java -ea kilim.tools.DumpClass classes/*.class | egrep -o 'L[^ ]+;' |
sort | uniq

-- or even --

java -ea kilim.tools.DumpClass classes/*.class by itself


Produces:
Exception in thread "main" java.lang.NoClassDefFoundError: kilim/tools/
DumpClass
Caused by: java.lang.ClassNotFoundException: kilim.tools.DumpClass

at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)

Could not find the main class: kilim.tools.DumpClass. Program will
exit.


I had previously tried it when you posted it the first time to no
avail. This time I toyed around with it for about 10 minutes and tried
calling it every way in the book I felt like and it still couldn't
find it for whatever reason (even though it's present in the directory
structure).

I would send you our project directories because I'm sure you could
probably solve the problem in under 30 minutes but the problem is that
we're dealing with some proprietary / sensitive data =\

[Sriram Srinivasan]

On Nov 15, 2010, at 12:04 AM, xiode04 wrote:

> From the root application directory, calling:
> java -ea kilim.tools.DumpClass classes/*.class | egrep -o 'L[^ ]+;' |
> sort | uniq
>
> -- or even --
>
> java -ea kilim.tools.DumpClass classes/*.class by itself
>

That's because kilim.jar is not in your classpath. Do one thing.
Instead of using the -cp flag, put all the jars and class directories
in the CLASSPATH environment.

[xiode04]

I thought the CLASSPATH environment was already being set by my build
script "build.sh":

export CLASSPATH=./classes:./lib/asm-all-2.2.3.jar:./lib/kilim.jar:./
lib/OAPI.jar:$CLASSPATH

Just to be safe, I modified my run.sh script as such:

export CLASSPATH=./classes:./lib/asm-all-2.2.3.jar:./lib/kilim.jar:./
lib/OAPI.jar:$CLASSPATH

java -ea kilim.tools.DumpClass classes/*.class

I also tried:
java -ea kilim.tools.DumpClass -d classes/*.class

And finally:
java -ea kilim.tools.DumpClass classes/*.class | perl -ne 'print $1,

"\n" if (/(L[^ ]+;)/);' | sort | uniq

./run.sh
Exception in thread "main" java.io.FileNotFoundException: classes/
*.class (No such file or directory)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.<init>(FileInputStream.java:106)
at java.io.FileInputStream.<init>(FileInputStream.java:66)
at kilim.tools.DumpClass.<init>(DumpClass.java:72)
at kilim.tools.DumpClass.main(DumpClass.java:63)

Although doing an "ls" from my "pwd" (root level application folder),
there's clearly a classes folder (containing kilim/ OAPI/ and
application/ which are all woven class files).


Another question I have: When I'm calling "java -ea
kilim.tools.Weaver" or something along those lines (of course with a
CLASSPATH defined) from the root application folder directory, is it
looking for that class file in ./classes/ or in the JAR in ./lib/ ?
Also, should every JAR in ./lib/ already be woven (for instance when
I'm putting my library's JAR into the application projects ./lib/)?
Finally, when compiling the application (obviously with dependencies
on the library mentioned above), should the library's class files that
it uses to compile be woven or regular?


Sorry to keep pestering you, I know I just asked a lot of
information. We're just really interested in getting Kilim to work
with our application because the sheer performance it offers is
amazing and we've written a pretty cool application that already
implements the message-passing Actor Model per the Kilim
documentation.

Perhaps we should have started on a smaller scale and tested as we
went ^_^

[jason...@gmail.com]

Hi,

I am now at a point where I will need to get Kilim up to asm 3.1 (at least)
What would be involved in getting that done?

I am trying to use hibernate and spring 3.1 in the same application
(different areas, but same classpath) as kilim and the asm causes
conflicts.

I guess I can try and create a separate classloader for compiling and
weaving - I already have the separate classloader, but because my
project is based on maven its going to be tricky to include both
versions of asm in the tar.gz produced by appassembler.

It would be far easier to just have one version of asm to have to deal with

Thanks
Jason