ERROR: Unknown entry kind: Array.newInstance

278 views
Skip to first unread message

Ganesha Upadhyaya

unread,
Jan 12, 2012, 1:10:31 PM1/12/12
to tamiflex...@googlegroups.com
Hi All,

I am trying to run dacapo using tamiflex agent using following document: http://code.google.com/p/tamiflex/wiki/DaCapoAndSoot
I am not able to get past 2nd step [using reflection log to perform spark analysis]. Following are the detailed steps and error. Please help me with this.

Step 1: java -javaagent:poa-2.0.0.0.jar -jar dacapo-9.12-bach.jar avrora -s small
Output:
TamiFlex Play-Out Agent Version 2.0.0.0
============================================================
===== DaCapo 9.12 avrora starting =====
===== DaCapo 9.12 avrora PASSED in 4247 msec =====

============================================================
TamiFlex Play-Out Agent Version 2.0.0.0
Found 56 new log entries.

Step2: java -Xmx3072m -cp soot-2.4.0.jar soot.Main -w -app -p cg.spark enabled -p cg reflection-log:out/refl.log -cp /usr/lib/jvm/java-6-openjdk/jre/lib/jce.jar:/usr/lib/jvm/java-6-openjdk/jre/lib/rt.jar:out -include org.apache. -include org.w3c. -main-class Harness -d sootified/avrora-small Harness 
Output:
Soot started on Thu Jan 12 12:06:01 CST 2012
Exception in thread "main" java.lang.RuntimeException: Line: 'Array.newInstance;java.lang.String[];java.util.Arrays.copyOf;2772;;'
at soot.Scene.addReflectionTraceClasses(Scene.java:991)
at soot.Scene.loadBasicClasses(Scene.java:955)
at soot.Scene.loadNecessaryClasses(Scene.java:1018)
at soot.Main.run(Main.java:167)
at soot.Main.main(Main.java:141)
Caused by: java.lang.RuntimeException: Unknown entry kind: Array.newInstance
at soot.Scene.addReflectionTraceClasses(Scene.java:988)
... 4 more

My objective is just to collect all class files without reflection in it, so that I can perform static analysis on them, what are the minimal steps I need to get it done?

Thanks
Ganesh

Eric Bodden

unread,
Jan 13, 2012, 2:40:25 AM1/13/12
to tamiflex...@googlegroups.com
Hello Ganesh.

Unfortunately, at the moment TamiFlex is a bit ahead of Soot. Soot
currently only supports reflective calls of the following kinds:

Class.forName
Class.newInstance
Constructor.newInstance
Method.invoke
Field.set*
Field.get*

You code contains a call to Array.newInstance, which Soot does not yet
know how to handle.

There are two options to this:
1) Modify Soot to handle this call. (A contribution would be greatly
appreciated here.)
2) Configure TamiFlex to not monitor Array.newInstance calls. This
would make Soot unsound in the respective call sites, of course.

To do the latter, you have to modify ~/.tamiflex/poa.properties. You
will find an entry...

transformations =\

de.bodden.tamiflex.playout.transformation.array.ArrayNewInstanceTransformation
\

de.bodden.tamiflex.playout.transformation.array.ArrayMultiNewInstanceTransformation
\

de.bodden.tamiflex.playout.transformation.clazz.ClassForNameTransformation
\

de.bodden.tamiflex.playout.transformation.clazz.ClassGetDeclaredFieldsTransformation
\
...

Remove all the lines corresponding to entries that Soot currently does
not support.

I will have to update the online documentation accordingly, I guess.

Let me know if that works for you.

Eric

--
Eric Bodden, Ph.D., http://bodden.de/
Head of Secure Software Engineering Group at EC SPRIDE
Principal Investigator in Secure Services at CASED
Tel: +49 6151 16-75422    Fax: +49 6151 16-72051
Room 3.2.14, Mornewegstr. 30, 64293 Darmstadt

Ganesha Upadhyaya

unread,
Jan 14, 2012, 1:18:36 AM1/14/12
to tamiflex...@googlegroups.com
Hi Eric,

Thanks for the immediate help. It worked after removing multiple transformations. I am trying to reproduce what you have reported in the tamiflex recent paper where I want to produce sound static callgraph when compared to dynamic callgraph. Another point where I'm kind of stuck is, producing dynamic callgraph using your jvmti agent. I have compiled the agent and used it while running avrora with small setting, and it takes too much time to produce the trace file with method CALL entries. I'm running following command:
java -agentpath:./target/tracer.so=trace.txt -jar ../../tamiflex/dacapo-9.12-bach.jar avrora -s small
Output: It did produce trace file of 14MB, but program doesn't terminate (I do expect this large file, as your reported files in the repository are that huge too). Do you suspect any problem in my execution? (I would like to mention that, I do have enough powerful machine to test these)

Thanks
Ganesh

Eric Bodden

unread,
Jan 16, 2012, 8:05:17 AM1/16/12
to tamiflex...@googlegroups.com
Hi Ganesh.

On 14 January 2012 07:18, Ganesha Upadhyaya <ganes...@gmail.com> wrote:
> Hi Eric,
>
> Thanks for the immediate help. It worked after removing multiple
> transformations. I am trying to reproduce what you have reported in the
> tamiflex recent paper where I want to produce sound static callgraph when
> compared to dynamic callgraph. Another point where I'm kind of stuck is,
> producing dynamic callgraph using your jvmti agent. I have compiled the
> agent and used it while running avrora with small setting, and it takes too
> much time to produce the trace file with method CALL entries. I'm running
> following command:
> java -agentpath:./target/tracer.so=trace.txt -jar
> ../../tamiflex/dacapo-9.12-bach.jar avrora -s small

It may well be the case that this takes really long. I remember that
for the experiments in our ICSE paper we ran the agent for several
days, actually. (On a fast compute server, that is.)

> Output: It did produce trace file of 14MB, but program doesn't terminate (I
> do expect this large file, as your reported files in the repository are that
> huge too). Do you suspect any problem in my execution? (I would like to
> mention that, I do have enough powerful machine to test these)

I guess it may actually terminate eventually, it may just take a
really long time. Note that usually it should not be necessary to run
the JVMTI agent. The point of our experiments was to do this *once* to
show that if tamiflex is used correctly, and all calls that tamiflex
monitors are taken into account, then this is sufficient to get a call
graph that is representative of all monitored runs.

Eric

Andreas Sewe

unread,
Jan 16, 2012, 8:35:07 AM1/16/12
to tamiflex...@googlegroups.com
Hi Ganesh,

as I was the one who wrote the JVMTI agent for our paper, here are my 2
cents:

>> Thanks for the immediate help. It worked after removing multiple
>> transformations. I am trying to reproduce what you have reported in the
>> tamiflex recent paper where I want to produce sound static callgraph when
>> compared to dynamic callgraph. Another point where I'm kind of stuck is,
>> producing dynamic callgraph using your jvmti agent. I have compiled the
>> agent and used it while running avrora with small setting, and it takes too
>> much time to produce the trace file with method CALL entries. I'm running
>> following command:
>> java -agentpath:./target/tracer.so=trace.txt -jar
>> ../../tamiflex/dacapo-9.12-bach.jar avrora -s small
>
> It may well be the case that this takes really long. I remember that
> for the experiments in our ICSE paper we ran the agent for several
> days, actually. (On a fast compute server, that is.)

Yes, the JVMTI agent was not designed to be fast (relatively verbose,
human-readable output; possibly high contention for a largish critical
section:
<http://code.google.com/p/tamiflex/source/browse/jvmti-tracer/src/tracer.c?repo=benchmarks#43>);
it was simply designed for a one-off experiment.

What it was designed for, however, was to be straight-forward. Maybe
some added printf-statements can give already you an idea where the
problem may be.

>> Output: It did produce trace file of 14MB, but program doesn't terminate (I
>> do expect this large file, as your reported files in the repository are that
>> huge too). Do you suspect any problem in my execution? (I would like to
>> mention that, I do have enough powerful machine to test these)
>
> I guess it may actually terminate eventually, it may just take a
> really long time.

Can you produce a thread-dump (with kill -3 $JAVA_PID) to see where it
hangs? I haven't experienced any deadlocks, personally, but that doesn't
mean there are none.

Hope this helps.

Andreas

rogerl...@gmail.com

unread,
May 7, 2013, 1:39:47 AM5/7/13
to tamiflex...@googlegroups.com
Hi Eric,

As you suggested, I comment out the transformations which soot cannot handle in ~/.tamiflex/poa.properties.
However, when I rerun the command "java -javaagent:poa-2.0.1.jar -jar dacapo-9.12-bach.jar avrora -s small". The exception occurs since tamiflex cannot find the class I commented out (e.g.,ArrayMultiNewInstanceTransformation,etc.) See the following exception description:
-==============================
TamiFlex Play-Out Agent Version 2.0.1
Loaded properties from /Users/roger/.tamiflex/poa.properties
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:323)
at sun.instrument.InstrumentationImpl.loadClassAndCallPremain(InstrumentationImpl.java:338)
Caused by: java.lang.RuntimeException: There was an error instantiating the instrument # de.bodden.tamiflex.playout.transformation.array.ArrayMultiNewInstanceTransformation
at de.bodden.tamiflex.playout.ReflectionMonitor.<init>(ReflectionMonitor.java:58)
at de.bodden.tamiflex.playout.Agent.instrumentClassesForLogging(Agent.java:235)
at de.bodden.tamiflex.playout.Agent.premain(Agent.java:105)
... 6 more
Caused by: java.lang.ClassNotFoundException: # de/bodden/tamiflex/playout/transformation/array/ArrayMultiNewInstanceTransformation
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:169)
at de.bodden.tamiflex.playout.ReflectionMonitor.<init>(ReflectionMonitor.java:46)
... 8 more
FATAL ERROR in native method: processing of -javaagent failed
Abort trap: 6
===================
So do I need to modify the source or something else to let it work?

Thanks

Roger

在 2012年1月13日星期五UTC+11下午6时40分25秒,Eric写道:

Eric Bodden

unread,
May 7, 2013, 2:12:38 AM5/7/13
to tamiflex...@googlegroups.com
Hi Roger.

This does not make much sense. If you commented out a class then TamiFlex should *not* be looking for this class any longer. Maybe there's an issue with whitespace or so in your config file?

Eric
> --
> You received this message because you are subscribed to the Google Groups "tamiflex-discuss" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to tamiflex-discu...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

rogerl...@gmail.com

unread,
May 7, 2013, 2:30:20 AM5/7/13
to tamiflex...@googlegroups.com, eric....@sit.fraunhofer.de
Hi Eric,

Yes, you are right. It works now. However, I found that soot-2.5.0 does not handle Field.get* either. 

Thanks a lot for your help.

Roger

在 2013年5月7日星期二UTC+10下午4时12分38秒,Eric Bodden写道:

Aritra Sengupta

unread,
Sep 14, 2013, 6:18:46 PM9/14/13
to tamiflex...@googlegroups.com, eric....@sit.fraunhofer.de
Hi Roger,
             I am facing a similar issue. In the post you mentioned the possibility of the issue being with white spaces in the properties file after the change.
Do you remember the fix ?

Thanks,
Aritra Sengupta.
PhD Student.
Dept of Computer Science & Engineering.
The Ohio State University.

Roger Lee

unread,
Sep 14, 2013, 9:54:17 PM9/14/13
to tamiflex...@googlegroups.com
Hi Aritra,

As far as I remembered, maybe due to the particular split resolution of string tokens, you'd better delete the whole sentence instead of commenting it using a "#". I do not know if you use the comment or not.

Hopefully it works for you.

Cheers,

Roger


--
You received this message because you are subscribed to a topic in the Google Groups "tamiflex-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/tamiflex-discuss/6q6ShId_TIw/unsubscribe.

To unsubscribe from this group and all its topics, send an email to tamiflex-discu...@googlegroups.com.

For more options, visit https://groups.google.com/groups/opt_out.



--

==================================================
   Yue Li (Roger Lee)                                                  
   ----------------------------                                                  
   PhD student of Compiler Research Group (CORG),    
   Computer Science and Engineering,                             
   University of New South Wales, Sydney, Australia.     
==================================================

Maria Jenkins

unread,
Dec 26, 2014, 12:54:19 PM12/26/14
to tamiflex...@googlegroups.com
Hi All,

I am having the same issue as Ganesha. I have tried fixing the white space but it is still searching for ArrayNewInstance. Any other suggestions for a fix?
Reply all
Reply to author
Forward
0 new messages