Re: openstack-nova build failure? -- xmlunit ?

67 views
Skip to first unread message

Alex Heneveld

unread,
Mar 2, 2012, 12:43:30 PM3/2/12
to Hogan, Dirk John, jclouds...@cloudsoftcorp.com, jclou...@googlegroups.com, Ghantasala, Sivakumar

Good spot, Dirk.  Thanks.

All-  did a check-in lose a pom dependency somewhere?  I think there have been some recent experiments with XmlUnit which is implicated in the error message...

XmlUnit btw looks like it could improve our tests.  Does anyone have experience with this they could share with the list?

Best
Alex


(That looks

On 02/03/2012 17:16, Hogan, Dirk John wrote:

Hi-

Could it be that the build in the openstack-nova broke overnight? Or am I doing something wrong? This was working last night at about 7 PM PST.

 

Thanks

 

Dirk

 

[INFO] Scanning for projects...

[WARNING]

[WARNING] Some problems were encountered while building the effective model for org.jclouds.labs:openstack-nova:bundle:1.5.0-SNAPSHOT

[WARNING] 'parent.relativePath' of POM org.jclouds:jclouds-project:1.5.0-SNAPSHOT (C:\shredder\my_documents\msl\jclouds\git\project\pom.xml) points at org.jclouds:jclouds-multi instead of org.sonatype.oss:oss-parent, please verify your project structure @ org.jclouds:jclouds-project:1.5.0-SNAPSHOT, C:\shredder\my_documents\msl\jclouds\git\project\pom.xml, line 24, column 13

[WARNING]

[WARNING] It is highly recommended to fix these problems because they threaten the stability of your build.

[WARNING]

[WARNING] For this reason, future Maven versions might no longer support building such malformed projects.

[WARNING]

[INFO]                                                                        

[INFO] ------------------------------------------------------------------------

[INFO] Building jcloud openstack-nova api 1.5.0-SNAPSHOT

[INFO] ------------------------------------------------------------------------

[INFO]

[INFO] --- maven-enforcer-plugin:1.0.1:enforce (enforce-maven) @ openstack-nova ---

[INFO]

[INFO] --- maven-enforcer-plugin:1.0.1:enforce (enforce-banned-dependencies) @ openstack-nova ---

[INFO]

[INFO] --- maven-enforcer-plugin:1.0.1:enforce (enforce-java) @ openstack-nova ---

[INFO]

[INFO] --- maven-remote-resources-plugin:1.2.1:process (process-remote-resources) @ openstack-nova ---

SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".

SLF4J: Defaulting to no-operation (NOP) logger implementation

SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.

[INFO]

[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ openstack-nova ---

[debug] execute contextualize

[INFO] Using 'UTF-8' encoding to copy filtered resources.

[INFO] skip non existing resourceDirectory C:\shredder\my_documents\msl\jclouds\git\labs\openstack-nova\src\main\clojure

[INFO] skip non existing resourceDirectory C:\shredder\my_documents\msl\jclouds\git\labs\openstack-nova\src\main\resources

[INFO] Copying 2 resources

[INFO]

[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ openstack-nova ---

[INFO] Nothing to compile - all classes are up to date

[INFO]

[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ openstack-nova ---

[debug] execute contextualize

[INFO] Using 'UTF-8' encoding to copy filtered resources.

[INFO] skip non existing resourceDirectory C:\shredder\my_documents\msl\jclouds\git\labs\openstack-nova\src\test\clojure

[INFO] Copying 13 resources

[INFO] Copying 2 resources

[INFO]

[INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ openstack-nova ---

[INFO] Compiling 1 source file to C:\shredder\my_documents\msl\jclouds\git\labs\openstack-nova\target\test-classes

[INFO]

[INFO] --- maven-surefire-plugin:2.12:test (default-test) @ openstack-nova ---

[INFO] Surefire report directory: C:\shredder\my_documents\msl\jclouds\git\labs\openstack-nova\target\surefire-reports

[INFO] Using configured provider org.apache.maven.surefire.testng.TestNGProvider

 

-------------------------------------------------------

T E S T S

-------------------------------------------------------

Running TestSuite

Configuring TestNG with: org.apache.maven.surefire.testng.conf.TestNGMapConfigurator@7d2a1e44

org.apache.maven.surefire.util.SurefireReflectionException: java.lang.reflect.InvocationTargetException; nested exception is java.lang.reflect.InvocationTargetException: null

java.lang.reflect.InvocationTargetException

       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

       at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)

       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

       at java.lang.reflect.Method.invoke(Method.java:597)

       at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)

       at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)

       at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)

       at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:103)

       at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:74)

Caused by: java.lang.NoClassDefFoundError: org/custommonkey/xmlunit/DifferenceListener

       at java.lang.Class.getDeclaredMethods0(Native Method)

       at java.lang.Class.privateGetDeclaredMethods(Class.java:2427)

       at java.lang.Class.privateGetPublicMethods(Class.java:2547)

       at java.lang.Class.getMethods(Class.java:1410)

       at org.testng.internal.TestNGClassFinder.<init>(TestNGClassFinder.java:60)

       at org.testng.TestRunner.initMethods(TestRunner.java:405)

       at org.testng.TestRunner.init(TestRunner.java:231)

       at org.testng.TestRunner.init(TestRunner.java:201)

       at org.testng.TestRunner.<init>(TestRunner.java:150)

       at org.testng.SuiteRunner$DefaultTestRunnerFactory.newTestRunner(SuiteRunner.java:523)

       at org.testng.SuiteRunner.init(SuiteRunner.java:157)

       at org.testng.SuiteRunner.<init>(SuiteRunner.java:111)

       at org.testng.TestNG.createSuiteRunner(TestNG.java:1245)

       at org.testng.TestNG.createSuiteRunners(TestNG.java:1232)

       at org.testng.TestNG.runSuitesLocally(TestNG.java:1086)

       at org.testng.TestNG.run(TestNG.java:1007)

       at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:76)

       at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.executeMulti(TestNGDirectoryTestSuite.java:161)

       at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:101)

       at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:115)

       ... 9 more

Caused by: java.lang.ClassNotFoundException: org.custommonkey.xmlunit.DifferenceListener

       at java.net.URLClassLoader$1.run(URLClassLoader.java:202)

       at java.security.AccessController.doPrivileged(Native Method)

       at java.net.URLClassLoader.findClass(URLClassLoader.java:190)

       at java.lang.ClassLoader.loadClass(ClassLoader.java:306)

       at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)

       at java.lang.ClassLoader.loadClass(ClassLoader.java:247)

       ... 29 more

 

Results :

 

Tests run: 0, Failures: 0, Errors: 0, Skipped: 0

 

[INFO] ------------------------------------------------------------------------

[INFO] BUILD FAILURE

[INFO] ------------------------------------------------------------------------

[INFO] Total time: 4.086s

[INFO] Finished at: Fri Mar 02 09:11:27 PST 2012

[INFO] Final Memory: 23M/223M

[INFO] ------------------------------------------------------------------------

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.12:test (default-test) on project openstack-nova: Error occurred in starting fork, check output in log -> [Help 1]

[ERROR]

[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.

[ERROR] Re-run Maven using the -X switch to enable full debug logging.

[ERROR]

[ERROR] For more information about the errors and possible solutions, please read the following articles:

[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

 


Adam Lowe

unread,
Mar 2, 2012, 1:52:26 PM3/2/12
to Hogan, Dirk John, jclou...@googlegroups.com, jclouds...@cloudsoftcorp.com, Ghantasala, Sivakumar
Dirk,

I added XMLUnit to project/pom.xml as a test dependency last night to assist with the vcloud director work - it should also help us to improve coverage of any other APIs we send XML message to. All the supported and labs modules including openstack-nova built last night, hence the change was accepted. 

Is it possible that you're building with an older version of project/pom.xml? Because of how maven snapshots jars get refreshed each day, occasionally a build will break till you pull the corresponding changes to pom.xmls from github.

If git pull doesn't resolve the problem, please let us know some more details of how your building openstack-nova.

Thanks,

Adam.

--
You received this message because you are subscribed to the Google Groups "jclouds-dev" group.
To post to this group, send email to jclou...@googlegroups.com.
To unsubscribe from this group, send email to jclouds-dev...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/jclouds-dev?hl=en.

Andrew Phillips

unread,
Mar 4, 2012, 6:34:25 AM3/4/12
to jclou...@googlegroups.com
> I added XMLUnit to project/pom.xml as a test dependency last night
> to assist with the vcloud director work - it should also help us to
> improve coverage of any other APIs we send XML message to. All the
> supported and labs modules including openstack-nova built last
> night, hence the change was accepted.

Just out of curiosity. why did you add the dependency to the project
POM instead of e.g. the POM for your provider/API or a "closer"
parent, e.g. a hypothetical vcloud-common?

How many projects that you're working on currently need the dependency?

Have a good Sunday, everyone!

ap

Adam Lowe

unread,
Mar 4, 2012, 8:11:26 AM3/4/12
to jclou...@googlegroups.com
Andrew,

Currently only labs/vcloud-director requires the change, but I moved the test dependency up as the HttpRequest comparison code I modified is in the core test-jar (BaseRestClientExpectTest). My hope is that making it easier for anyone to write expect tests for providers that requires us to PUT or POST XML to an API server would prove enough of a win to justify making the change globally.

So long as vcloud-director (and all the other vcloud projects) retain the XMLUnit test dependency and have a common XMLUnit-enabled BaseRestClientExpectTest, I don't really mind where it comes from. We could certainly create a maven 'xml-common' module for this, my only concern is that we'd be complicating the structure of the jclouds project for relatively little gain.

Thanks,

Adam.

Adrian Cole

unread,
Mar 4, 2012, 8:50:49 AM3/4/12
to jclou...@googlegroups.com

Hey, Adam.

Sounds like this could be handy in opsource which Kedar is working on.  Mind pasting a snippet of the xml assertion you are using?

-A

Andrew Phillips

unread,
Mar 4, 2012, 10:04:13 AM3/4/12
to jclou...@googlegroups.com
> So long as vcloud-director (and all the other vcloud projects)
> retain the XMLUnit test dependency and have a common XMLUnit-enabled
> BaseRestClientExpectTest, I don't really mind where it comes from.
> We could certainly create a maven 'xml-common' module for this, my
> only concern is that we'd be complicating the structure of the
> jclouds project for relatively little gain.

Clear, thanks for the explanation! Obviously if the core test-jar is
modified it makes sense to have the dependency at least there...

ap

Adam Lowe

unread,
Mar 4, 2012, 10:12:21 AM3/4/12
to jclou...@googlegroups.com
Folks,

The thing that got me excited about this is that XMLUnit's Diff.identical() checks equality correctly: that the correct elements are all present and in order; each element's attributes have the correct values; everything is in the correct namespace; crucially it ignores attribute order and meaningless whitespace.

I'll walk through how to configure the expect tests first as they also unit test much of the client wiring (guice) - the actual XMLUnit code is towards the end of this message.

The XML request parsing for vcloud-director expect tests is enabled by the following - in this case all non-empty request payloads are compared as XML, you could use Content-type, etc. to decide:-


----- vcloud-director/BaseVCloudDirectorRestClientExpectTest.java

   @Override
   public HttpRequestComparisonType compareHttpRequestAsType(HttpRequest input) {
      if (input.getPayload() == null || input.getPayload().getContentMetadata().getContentLength() == 0) {
         return HttpRequestComparisonType.DEFAULT;
      }
      return HttpRequestComparisonType.XML;
   }

------


I've trimmed this down a little, for the real thing have a look at VAppTemplateClientExpectTest in labs/vcloud-director. The key with this snippet is that we pass a domain object, built up by the appropriate builder, into the editVAppTemplate method:


----- vcloud-director/VAppTemplateClientExpectTest.java

      final String templateId = "/vAppTemplate/vAppTemplate/vappTemplate-xxxx-xxxx-xxxx-xxx";
      Reference vappTemplateRef = Reference.builder().href(URI.create(endpoint + templateId)).build();

      VAppTemplateClient client = orderedRequestsSendResponses(loginRequest, sessionResponse,
            new VcloudHttpRequestPrimer()
.apiCommand("PUT", templateId).xmlFilePayload("/vapptemplate/vAppTemplate.xml",VAPP_TEMPLATE)
                .acceptMedia(TASK).httpRequestBuilder().build(),
            new VcloudHttpResponsePrimer().xmlFilePayload("/task/task.xml", TASK).httpResponseBuilder().build(),
            ).getVAppTemplateClient();

     VAppTemplate template =
                VAppTemplate.builder().href(URI.create("https://vcloudbeta.bluelock.com/api/vAppTemplate/vappTemplate-xxxx"))
            .links(ImmutableSet.of(aLink, bLink))
            .children(ImmutableSet.<VAppTemplate>of())
// ...
            .ovfDescriptorUploaded(true)
            .goldMaster(false)
            .build();
                  
      Task task = client.editVAppTemplate(vappTemplateRef, template);
      assertNotNull(task);

------


Without XMLUnit the way to get this test to pass would be to adjust the whitespace in "vapptemplate/vAppTemplate.xml" to match precisely the same string JAXB is generating (which is either tedious or, worse, a dangerous copy-and-paste from test output to test input!).

Note that I've configured the expect test scenario to ignore some XML metadata that we still haven't convinced JAXB to reproduce, even with the correct annotations in package-info.java:
- although the correct namespaces are correctly applied, JAXB is using a different prefix for them (ns1, ns2, etc).
- schema location is lost


----- core/BaseRestClientExpectTest.java

public boolean httpRequestsAreEqual(HttpRequest a, HttpRequest b) {
// ... if compare as XML
Diff diff = XMLUnit.compareXML(Strings2.toStringAndClose(a.getPayload().getInput()),
                                              Strings2.toStringAndClose(b.getPayload().getInput()));
               
// Ignoring xsi:schemaLocation and differences in namespace prefixes
diff.overrideDifferenceListener(new DifferenceListener() {
   @Override
    public int differenceFound(Difference difference) {
          if (difference.getId() == DifferenceConstants.SCHEMA_LOCATION_ID ||
              difference.getId() == DifferenceConstants.NAMESPACE_PREFIX_ID) {
                   return RETURN_IGNORE_DIFFERENCE_NODES_IDENTICAL;
         }
         return RETURN_ACCEPT_DIFFERENCE;
   }

  @Override
  public void skippedComparison(Node node, Node node1) {
  }
});
               
return diff.identical() && Objects.equal(a.getHeaders(), b.getHeaders());
//...handle other types of HttpRequest

-----


It's also worth noting that when objects aren't identical the Diff contains details of what's different. For now we can debug to this point to see what's up, but it would be really good to expose this information to the developer when an expect test fails rather than (or in addition to) rendering the request and all the expected requests (even better a DetailedDiff with all the differences listed).

Thanks,

Adam.

Koper, Dies

unread,
Mar 4, 2012, 6:03:38 PM3/4/12
to jclou...@googlegroups.com

Hi Adam,

 

As I am receiving XML responses from the FGCP API I am wondering whether XMLUnit would be useful for my tests too.

 

Ø  Without XMLUnit the way to get this test to pass would be to adjust the whitespace in "vapptemplate/vAppTemplate.xml" to match precisely the same string JAXB is generating (which is either tedious or, worse, a dangerous copy-and-paste from test output to test input!).

 

Is the issue with whitespaces the only reason why you can’t rely on normal diffing?

Or are there other issues, like attribute order (can they change from time to time, or maybe when using a different JDK[1])?

 

I’ve been copying and pasting from test output to test input, or from live test responses (with a bit of manual editing of user data). I haven’t encountered any issues with that yet.

As long as I have live tests for the same use case, and they passed once, I am assuming that when my equivalent expect tests pass, everything is OK.

 

[1] I’ve been hearing of similar issues with JDK 7.x where Class.getMethods and related reflection methods, although documented to return results in an undefined order, have always returned the same order for all JDK versions until recently, breaking some people’s code.

 

Cheers,

Dies Koper

Reply all
Reply to author
Forward
0 new messages