Re: Same output in pre-fix and post-fix
Valentin Dallmeier
Sorry that you're still having problems. I took a look at
http://www.ics.uci.edu/~fservant/public/candidatetests.csv and saw
many instances of
Exception in thread ""main"" java.lang.NoClassDefFoundError:
org/aspectj/testing/drivers/Harness
This usually happens if the tests are not built correctly. Can you
tell me what you did before you ran the tests?
Regards,
Valentin
On Tue, Sep 28, 2010 at 7:41 AM, Francisco Servant <fser...@ics.uci.edu> wrote:
> Hello Valentin,
>
> I have picked some sample bug numbers throughout the history of
> AspectJ for which there are test cases in post-fix which do not exist
> in pre-fix.
>
> I use the following heuristic to copy these tests (the ones that only
> exist in post-fix) to the pre-fix version:
> -If it is inside a .java file, copy the .java file over
> -If there is at least one of these test cases inside a .xml file,
> delete the contents of "pre-fix/org.aspectj/modules/tests" and make
> "pre-fix/org.aspectj/modules/tests" link to "post-fix/org.aspectj/
> modules/tests"
>
> After this, I build and execute these tests in pre-fix and post-fix.
> However, all of them produce the same behavior (output) for both pre-
> fix and post-fix. Could you help me find what I am doing wrong?
>
> Thank you!
> Francisco
>
> P.S.:
> More details about the executions can be found in the files:
> http://www.ics.uci.edu/~fservant/public/candidatetests.csv
> http://www.ics.uci.edu/~fservant/public/candidatetests.xls
>
> The list of bug ids, test locations and test names that I have used is
> below:
> 33635 org.aspectj/modules/tests/jimTests.xml Negation of if pointcut
> does not work
> 53981 org.aspectj/modules/tests/ajcTests.xml proceed used as method
> name in around advice (1)
> 53981 org.aspectj/modules/tests/ajcTests.xml proceed used as method
> name in around advice (2)
> 53981 org.aspectj/modules/tests/ajcTests.xml proceed used as method
> name in around advice (3)
> 53981 org.aspectj/modules/tests/src/org/aspectj/systemtest/ajc121/
> ajc121-tests.xml proceed used as method name in around advice (1)
> 53981 org.aspectj/modules/tests/src/org/aspectj/systemtest/ajc121/
> ajc121-tests.xml proceed used as method name in around advice (2)
> 53981 org.aspectj/modules/tests/src/org/aspectj/systemtest/ajc121/
> ajc121-tests.xml proceed used as method name in around advice (3)
> 53981 org.aspectj/modules/tests/src/org/aspectj/systemtest/ajc121/
> Ajc121Tests.java test023_proceedInAround1
> 53981 org.aspectj/modules/tests/src/org/aspectj/systemtest/ajc121/
> Ajc121Tests.java test024_proceedInAround2
> 53981 org.aspectj/modules/tests/src/org/aspectj/systemtest/ajc121/
> Ajc121Tests.java test025_proceedInAround3
> 58520 org.aspectj/modules/ajdoc/testsrc/org/aspectj/tools/ajdoc/
> FullyQualifiedArgumentTest.java testPr58520
> 82134 org.aspectj/modules/tests/src/org/aspectj/systemtest/ajc150/
> MigrationTests.java testMigrationFrom120_pointcutsAndAdvice
> 82134 org.aspectj/modules/tests/src/org/aspectj/systemtest/ajc150/
> MigrationTests.java testMigrationFrom121_pointcutsAndAdvice
> 132349 org.aspectj/modules/tests/src/org/aspectj/systemtest/ajc151/
> ajc151.xml @AJ deow doesn't throw an invalidAbsoluteTypeName when
> specify type in the same file
> 132349 org.aspectj/modules/tests/src/org/aspectj/systemtest/ajc151/
> ajc151.xml @AJ deow doesn't throw an invalidAbsoluteTypeName when
> specify type in the same package
> 132349 org.aspectj/modules/tests/src/org/aspectj/systemtest/ajc151/
> ajc151.xml ITD on inner type of generic type
> 132349 org.aspectj/modules/tests/src/org/aspectj/systemtest/ajc151/
> Ajc151Tests.java testAtAspectNoInvalidAbsoluteTypeName_pr126560
> 132349 org.aspectj/modules/tests/src/org/aspectj/systemtest/ajc151/
> Ajc151Tests.java testAtAspectNoInvalidAbsoluteTypeName_pr126560_2
> 132349 org.aspectj/modules/tests/src/org/aspectj/systemtest/ajc151/
> Ajc151Tests.java testItdOnInnerTypeOfGenericType_pr132349
> 132926 org.aspectj/modules/tests/src/org/aspectj/systemtest/ajc151/
> ajc151.xml crashing on annotation type resolving with asm - 1
> 132926 org.aspectj/modules/tests/src/org/aspectj/systemtest/ajc151/
> ajc151.xml crashing on annotation type resolving with asm - 2
> 132926 org.aspectj/modules/tests/src/org/aspectj/systemtest/ajc151/
> ajc151.xml crashing on annotation type resolving with asm - 3
> 132926 org.aspectj/modules/tests/src/org/aspectj/systemtest/ajc151/
> ajc151.xml doubly annotating a method with declare
> 132926 org.aspectj/modules/tests/src/org/aspectj/systemtest/ajc151/
> Ajc151Tests.java testCrashingWithASM_pr132926_1
> 132926 org.aspectj/modules/tests/src/org/aspectj/systemtest/ajc151/
> Ajc151Tests.java testCrashingWithASM_pr132926_2
> 132926 org.aspectj/modules/tests/src/org/aspectj/systemtest/ajc151/
> Ajc151Tests.java testCrashingWithASM_pr132926_3
> 132926 org.aspectj/modules/tests/src/org/aspectj/systemtest/ajc151/
> Ajc151Tests.java testDeca
> 145693 org.aspectj/modules/tests/src/org/aspectj/systemtest/ajc153/
> Ajc153Tests.java testAdviceNotWovenAspectPath_pr147841
> 145693 org.aspectj/modules/tests/src/org/aspectj/systemtest/ajc153/
> Ajc153Tests.java testAtajInheritance_pr149305_1
> 145693 org.aspectj/modules/tests/src/org/aspectj/systemtest/ajc153/
> Ajc153Tests.java testAtajInheritance_pr149305_2
> 145693 org.aspectj/modules/tests/src/org/aspectj/systemtest/ajc153/
> Ajc153Tests.java testAtajInheritance_pr149305_3
> 145693 org.aspectj/modules/tests/src/org/aspectj/systemtest/ajc153/
> Ajc153Tests.java testCpathNoTypeCflowField_pr145693_3
> 145693 org.aspectj/modules/tests/src/org/aspectj/systemtest/ajc153/
> Ajc153Tests.java testIncorrectAnnotationValue_pr148537
> 145693 org.aspectj/modules/tests/src/org/aspectj/systemtest/ajc153/
> Ajc153Tests.java testVerifyErrInpathNoTypeCflowField_pr145693_2
> 145693 org.aspectj/modules/tests/src/org/aspectj/systemtest/ajc153/
> Ajc153Tests.java testVerifyErrNoTypeCflowField_pr145693_1
> 145693 org.aspectj/modules/tests/src/org/aspectj/systemtest/ajc153/
> Ajc153Tests.java testVisibilityProblem_pr149071
>
> --
> You received this message because you are subscribed to the Google Groups "iBugs" group.
> To post to this group, send email to ib...@googlegroups.com.
> To unsubscribe from this group, send email to ibugs+un...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/ibugs?hl=en.
>
>
Francisco Servant
Francisco Servant
You are right. It seems like I was not building the test cases
correctly. At least for bug #53981 so far...
The problem is that the process of building the test cases can fail.
This problem is solved by executing the 'buildtests' script again, as
we did for building the versions.
However, the difference is that when the process of building a version
fails, we get a message in stderr, whereas for the process of building
the tests, we need to check stdout to find the error message.
Anyways, I am fixing my code and let you know if I find more problems.
Thank you!!
Francisco
On Sep 28, 6:39 am, Valentin Dallmeier <valentin.dallme...@gmail.com>
wrote:
> Hi Francisco!
>
> Sorry that you're still having problems. I took a look athttp://www.ics.uci.edu/~fservant/public/candidatetests.csvand saw
>
> Exception in thread ""main"" java.lang.NoClassDefFoundError:
> org/aspectj/testing/drivers/Harness
>
> This usually happens if the tests are not built correctly. Can you
> tell me what you did before you ran the tests?
>
> Regards,
>
> Valentin
>
Francisco Servant
So I fixed the problem and now I do get different behavior in pre-fix
and post-fix for .xml tests.
However, all of the .java tests that existed in post-fix and not in
pre-fix (and that I copied over to pre-fix) still return
java.lang.NoClassDefFoundError errors in both pre-fix and post-fix.
Can you take a look at http://www.ics.uci.edu/~fservant/public/candidatetests2.csv
and see if this is normal behavior?
Thanks!
Francisco
Valentin Dallmeier
That's definitely not normal. It looks like the generated test proxy
is unable to find the original test. If you did not touch the class
path, this usually means that the tests did not build correctly. Can
you post the process that you use to build the version and the tests?
Regards,
Valentin
Francisco Servant
This is the process that I follow, starting from scratch:
ant -DfixId=58520 checkoutversion
ant -DfixId=58520 -Dtag=pre-fix buildversion
ant -DfixId=58520 -Dtag=post-fix buildversion
ant -DfixId=58520 -Dtag=pre-fix buildtests
ant -DfixId=58520 -Dtag=post-fix buildtests
ant -DfixId=58520 -Dtag=post-fix -DtestFileName=" org.aspectj/modules/
ajdoc/testsrc/org/aspectj/tools/ajdoc/FullyQualifiedArgumentTest.java"
-DtestName="testPr58520" -DjvmArgs="-Xmx1024m" gentestscript
cd versions/58520/post-fix/org.aspectj/modules/tests
ant -f runtest.xml
And I get the result:
Buildfile: runtest.xml
runtest:
[java] .F
[java] Time: 0.002
[java] There was 1 failure:
[java] 1) warning(junit.framework.TestSuite
$1)junit.framework.AssertionFailedError: Exception in constructor:
testPr58520 (java.lang.NoClassDefFoundError: org/aspectj/tools/ajdoc/
FullyQualifiedArgumentTest
[java] at
org.aspectj.tools.ajdoc.FullyQualifiedArgumentTestProxy.<init>(org/
aspectj/tools/ajdoc/FullyQualifiedArgumentTest.java:24)
[java] at
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
[java] at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:
39)
[java] at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:
27)
[java] at
java.lang.reflect.Constructor.newInstance(Constructor.java:513)
[java] at
org.aspectj.tools.ajdoc.FullyQualifiedArgumentTestProxy.main(org/
aspectj/tools/ajdoc/FullyQualifiedArgumentTest.java:53)
[java] )
[java] at
org.aspectj.tools.ajdoc.FullyQualifiedArgumentTestProxy.main(org/
aspectj/tools/ajdoc/FullyQualifiedArgumentTest.java:53)
[java]
[java] FAILURES!!!
[java] Tests run: 1, Failures: 1, Errors: 0
[java]
BUILD SUCCESSFUL
Total time: 0 seconds
However, there are some .java test cases that do work, like:
cd ../../../../../..
ant -DfixId=58520 -Dtag=post-fix -DtestFileName="org.aspectj/modules/
ajde/testsrc/org/aspectj/ajde/StructureModelTest.java" -
DtestName="testMainClassNodeInfo" -DjvmArgs="-Xmx1024m" gentestscript
cd versions/58520/post-fix/org.aspectj/modules/tests
ant -f runtest.xml
returning:
Buildfile: runtest.xml
runtest:
[java] .
[java] Time: 1.562
[java]
[java] OK (1 test)
[java]
BUILD SUCCESSFUL
Total time: 3 seconds
The full list of my results for the post-fix test cases for bug 58520
is in file: http://www.ics.uci.edu/~fservant/public/javaTests.csv
Don't mind the errors caused by the DISPLAY variable not being set. I
have already fixed that :)
Thank you again!
Francisco
On Sep 30, 12:53 am, Valentin Dallmeier <valentin.dallme...@gmail.com>
wrote:
>
> That's definitely not normal. It looks like the generated test proxy
> is unable to find the original test. If you did not touch the class
> path, this usually means that the tests did not build correctly. Can
> you post the process that you use to build the version and the tests?
>
> Regards,
>
> Valentin
>
> > Hi again,
>
> > So I fixed the problem and now I do get different behavior in pre-fix
> > and post-fix for .xml tests.
> > However, all of the .java tests that existed in post-fix and not in
> > pre-fix (and that I copied over to pre-fix) still return
> > java.lang.NoClassDefFoundError errors in both pre-fix and post-fix.
>
Valentin Dallmeier
I reproduced the problem. The issue is that, the ajdoc jar is not on
the class path. A quick fix is to add the following line
<pathelement
location="${moduledir}/aj-build/jars/ajdoc-test-all.jar"/>
to the generated runtest.xml . However, this works only for this (and
other ajdoc) tests.
Regards,
Valentin
Francisco Servant
I cannot reproduce the solution. After taking these steps:
ant -DfixId=58520 checkoutversion
ant -DfixId=58520 -Dtag=post-fix buildversion
ant -DfixId=58520 -Dtag=post-fix buildtests
I cannot find the file ajdoc-test-all.jar in "versions/58520/post-fix/
org.aspectj/modules/aj-build/jars" or in any other path.
Do I have to compile ajdoc separately?
Thank you,
Francisco
On Oct 2, 3:43 am, Valentin Dallmeier <valentin.dallme...@gmail.com>
wrote:
> Hi Francisco!
>
> the class path. A quick fix is to add the following line
>
> <pathelement
> location="${moduledir}/aj-build/jars/ajdoc-test-all.jar"/>
>
> to the generated runtest.xml . However, this works only for this (and
> other ajdoc) tests.
>
> Regards,
>
> Valentin
>
>
> read more »
Valentin Dallmeier
and run "ant test". Does that help?
Francisco Servant
Now, the problem is that this only solves the problem for ajdoc test
cases.
Do you recommend separately compiling ajdoc and adding it to all test
cases? Could this produce undesired results?
If it is ok to compile ajdoc and add it to the class path for all test
cases, shall I also do the same for other modules for which I am
receiving the same error message?
I executed all the tests for some sample bug ids and I got
NoClassDefFoundError for classes in these packages:
org/aspectj/build
org/aspectj/internal/build
org/aspectj/internal/tools/build
org/aspectj/runtime/reflect
org/aspectj/systemtest/ajc121
org/aspectj/systemtest/ajc150
org/aspectj/systemtest/ajc151
org/aspectj/systemtest/ajc153
org/aspectj/tools/ajbrowser
org/aspectj/tools/ajdoc
org/aspectj/util
However, this is not a comprehensive list. Do you know how I can
easily (without running all test cases in all versions) get the full
list of modules that I would need to build separately?
Would you recommend otherwise ignoring these test cases (with
NoClassDefFoundError) or considering them as 'failed'?
I am sorry I ended up asking a lot of questions :S
Let me know what you think.
Thank you again,
Francisco
On Oct 6, 1:06 am, Valentin Dallmeier <valentin.dallme...@gmail.com>
wrote:
> and run "ant test". Does that help?
>
>
> read more »
Valentin Dallmeier
Sorry that you're having troubles with the other modules, but you're
the first who's interested in testing them. My thoughts are as
follows:
- It is okay to separately build additional modules.
- You can add additional jars to the class path. Currently, there is
no easy way to do this, since the test case generator is Java code
which needs to be compiled against 1.4 only. To circumvent touching
this code, I would recommend adding additional jars to the class path
with ant. Maybe one of the tasks at
http://ant.apache.org/manual/tasksoverview.html#file is helpful for
that. Whatever you do, make sure that the new jars are added after the
ones that are already in the test, since those are the main jars.
I would be really happy if you could try to change the build.xml file
to incorporate your changes and send it back to me. Let me know if you
have further questions.
Regards,
Valentin