Re: Reference Application > Distribution > #1923 has FAILED (1 tests failed, no failures were new)

[Michael Seaton]

We have been battling a failure due to the latest reporting commit in our Mirebalis CI, but I've been baffled so far.

Basically, in the latest reporting module, we changed the OpenMRS version we build against in the POM from 1.6.6 to 1.9.7, and this seems to cause 2 new jars to be included in the lib folder of the build reporting module omod - xml-apis and log4j.  It is the xml-apis jar specifically that is causing our issue I think.  Nothing substantive outside of the OpenMRS version change seems to have been done that would cause this.

If anyone has any insight into why this is happening, it would be great to hear.  These jars show up as provided in the dependencies view in Intellij.  Running mvn dependency:tree also shows them as provided:

...
[INFO] |  +- xerces:xercesImpl:jar:2.8.0:provided
[INFO] |  +- xml-apis:xml-apis:jar:1.3.03:provided
[INFO] |  +- xml-resolver:xml-resolver:jar:1.1:provided
[INFO] |  +- javax.validation:validation-api:jar:1.0.0.GA:provided
...

However, running mvn clean package -X shows this in it's debug output:

...
[DEBUG]       org.openmrs.liquibase.ext:identity-insert:jar:1.2.1:provided
[DEBUG]       org.openmrs.liquibase.ext:type-converter:jar:1.0.1:provided
[DEBUG]       xerces:xercesImpl:jar:2.8.0:provided
[DEBUG]       xml-apis:xml-apis:jar:1.3.03:compile
[DEBUG]       xml-resolver:xml-resolver:jar:1.1:provided
[DEBUG]       javax.validation:validation-api:jar:1.0.0.GA:provided
[DEBUG]       org.hibernate:hibernate-validator:jar:4.2.0.Final:provided
...

Any thoughts from maven gurus out there as to what's going on?

Thanks,
Mike

On 04/16/2014 07:01 PM, Wyclif Luyima wrote:

I have noticed reporting UI module is not starting.

@Mike, @Darius, do you happen to know why?

Wyclif

On Wed, Apr 16, 2014 at 6:31 PM, Wyclif Luyima <wyc...@openmrs.org> wrote:

When i run VisitTest.startTest() locally against devtest01, i see some errors in the UI see the attached stacktrace, i wonder what has changed.

Wyclif

On Wed, Apr 16, 2014 at 6:20 PM, Lee Breisacher <leebre...@gmail.com> wrote:

I'm traveling... will have a look at it later tonight when I get home.

Lee

On Apr 16, 2014 3:45 PM, "OpenMRS CI" <ci-no...@openmrs.org> wrote:

Reference Application › Distribution › #1923 failed Manual run by Wyclif Luyima 1/5 jobs failed with 1 failing test. Responsible No one is responsible for this build. Failing Jobs Job Duration Tests Run UI Tests (Create Standalone and Run UI Tests) 4 minutes 1 of 7 failed Logs | Artifacts Tests View full test details 1 Existing Test Failures Test Job VisitTest testStartVisit Run UI Tests View Online | Add Comments This message was sent by Atlassian Bamboo. If you wish to stop receiving these emails edit your user profile or notify your administrator.

--
Reference App Development: http://go.openmrs.org/dev-refapp
Post: dev-r...@openmrs.org
Unsubscribe: dev-refapp+...@openmrs.org
Manage your OpenMRS subscriptions at https://id.openmrs.org/
---
You received this message because you are subscribed to the Google Groups "Reference App Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dev-refapp+...@openmrs.org.
For more options, visit https://groups.google.com/a/openmrs.org/d/optout.

--
Reference App Development: http://go.openmrs.org/dev-refapp
Post: dev-r...@openmrs.org
Unsubscribe: dev-refapp+...@openmrs.org
Manage your OpenMRS subscriptions at https://id.openmrs.org/
---
You received this message because you are subscribed to the Google Groups "Reference App Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dev-refapp+...@openmrs.org.
For more options, visit https://groups.google.com/a/openmrs.org/d/optout.

--
Wyclif Luyima
Regenstrief Institute Inc.

Confidentiality Notice: The contents of this message and any files transmitted with it may contain confidential and/or privileged information and are intended solely for the use of the named addressee(s). Additionally, the information contained herein may have been disclosed to you from medical records with confidentiality protected by federal and state laws. Federal regulations and State laws prohibit you from making further disclosure of such information without the specific written consent of the person to whom the information pertains or as otherwise permitted by such regulations. A general authorization for the release of medical or other information is not sufficient for this purpose.

If you have received this message in error, please notify the sender by return e-mail and delete the original message. Any retention, disclosure, copying, distribution or use of this information by anyone other than the intended recipient is strictly prohibited.

--
Wyclif Luyima
Regenstrief Institute Inc.

Confidentiality Notice: The contents of this message and any files transmitted with it may contain confidential and/or privileged information and are intended solely for the use of the named addressee(s). Additionally, the information contained herein may have been disclosed to you from medical records with confidentiality protected by federal and state laws. Federal regulations and State laws prohibit you from making further disclosure of such information without the specific written consent of the person to whom the information pertains or as otherwise permitted by such regulations. A general authorization for the release of medical or other information is not sufficient for this purpose.

If you have received this message in error, please notify the sender by return e-mail and delete the original message. Any retention, disclosure, copying, distribution or use of this information by anyone other than the intended recipient is strictly prohibited.

[Wyclif Luyima]

Hi Mike,

Okay, that now explains the logs in the stacktrace i attached to my previous email.

 

When i look at the pom files in reporting, i don't see a dependency for xml-apis, reporting comes with the poi library which in turn comes with xml-apis but excludes it via the exclusions which should be fine. Seems to me like it is another module adding it which we need to find.

Wyclif

--
OpenMRS Developers: http://go.openmrs.org/dev
Post: d...@openmrs.org | Unsubscribe: dev+uns...@openmrs.org

Manage your OpenMRS subscriptions at https://id.openmrs.org/

[Michael Seaton]

(changing subject line)

Mark Goodrich did some good sleuthing and found a couple of things.

1. This article describes a similar issue, and links to several JIRA tickets -  MNG-5255,  MNG-5273 for maven core, both of which were closed as won't fix/can't reproduce in favor of a ticket in the maven war plugin project: MWAR-111,  So, there is some evidence here that this might be a bug in Maven.  Another comment on the article states that this surfaced in Maven 3, and that Maven 2 works correctly.  So I tried this out, and indeed when I try to package up the reporting omod using maven 2.2.1 the build succeeds and the correct dependent jars are included in the omod. 

We are not using the Maven war plugin as far as I know, but rather have a custom-built Maven plugin for building an omod, the source of which is in OpenMRS subversion.

My suspicion is that something changed in the Maven API between version 2 and version 3, and there are some edge cases that are not handled correctly (since things generally seem to work right but consistently do not for a repeatable set of things), and both our omod maven plugin and the Maven war plugin are including some incorrect dependencies as as result.

2. As a workaround / solution, Mark discovered that we can add an exclusion of xml-apis to both the openmrs-api dependency and the openmrs-api test dependency, and the result is that xml-api.jar is no longer included in the built omod.  log4j is still there for some reason, which is another thing to look into if it causes issues.  So we have taken approach #2 for now, and this seems to have fixed the issue on our end - CI now passes.

It'd still be great to dive deeper into #1 and find a solution though, if anyone has any thoughts.

Mike

To unsubscribe from this group and stop receiving emails from it, send an email to dev+uns...@openmrs.org.

[Cintia Del Rio]

I have pretty much no context about the modules (and omod itself), but it's well known that maven 3 changed the dependency resolution:

https://cwiki.apache.org/confluence/display/MAVEN/Maven+3.x+Compatibility+Notes#Maven3.xCompatibilityNotes-DependencyResolution

In theory, maven 3 is the correct one. In real life, I haven't seen more than 2 or 3 differences until now - and the worst case was a custom packaging plugin (similar to omod, I guess). 

--

-------
Sent from TARDIS. Typos might be a timey whyney thingy.
Enviado da TARDIS, podem existir erros devido à diferenças de espaço-tempo.
 
Cintia Del Rio

[Cintia Del Rio]

This is the issue I talked about: https://ecosystem.atlassian.net/browse/AMPS-1093
But Iit's not the same problem, as that plugin is using maven-bundle-plugin, and your omod plugin is only using @requiresDependencyResolution runtime from maven.

So, I guess this is the goal that actually building the omod:
http://svn.openmrs.org/openmrs-contrib/maven/plugins/maven-openmrs-plugin/trunk/src/main/java/org/motech/openmrs/plugin/PackageModuleMojo.java

The important bits are:

           @requiresDependencyResolution runtime

           ScopeArtifactFilter filter = new ScopeArtifactFilter( Artifact.SCOPE_RUNTIME );
            Iterator artifacts = project.getArtifacts().iterator();
            while ( artifacts.hasNext() )
            {
                Artifact candidate = (Artifact) artifacts.next();
                if ( !candidate.isOptional() && filter.include( candidate ) )
                {
                    getLog().debug( "bundling artifact: " + candidate.getArtifactId() );
                    FileUtils.copyFile( candidate.getFile(), new File( libDir, candidate.getFile().getName() ) );
                }
            }

Looking at the documentation, it shouldn't have changed:
https://maven.apache.org/developers/mojo-api-specification.html

Can you please give me the commit hash and repository? I might be able to debug it this weekend and say what the plugin is actually receiving.

[Michael Seaton]

Hi Cintia,

Thanks so much for the help.  The repository in question is here:

https://github.com/openmrs/openmrs-module-reporting

This is the commit I made yesterday to get things to work.  If these excludes are not added (and I don't think they should be needed here, since this dependency is listed as provided), then they are included in the packaged omod in the lib folder:

https://github.com/openmrs/openmrs-module-reporting/commit/0e989f4d14287dd7b22822b783f3a17d4cfc946d

Cheers,
Mike

[Lee Breisacher]

I see that this is working now (the RefApp Distro build is green). Can someone briefly summarize what the problem and fix was? Just curious...

Thanks,

Lee

To unsubscribe from this group and stop receiving emails from it, send an email to dev+uns...@openmrs.org.

[Wyclif Luyima]

Lee,

Did you see Mike's email? There was an xml-api dependency duplication that was causing class loading issues. Once they removed the duplicate library, the problem got resolved.

Wyclif

[Cintia Del Rio]

So, I guess you were trying to run "mvn dependency:tree" using maven < 3.1.0. The default dependency plugin version was 2.1, which didn't include the fix to show maven 3 dependency resolution. So, what you had was actually the exactly tree generated by maven 2.x (not maven 3.x).

(Friendly warning, even version 2.8 of the plugin will show maven 2 resolution if -Dverbose=true is turned on. I raised an issue for that).

So, running the latest version of dependency tree (2.8, regardless of which maven 3.x) in the omod reporting project will show the following non-provided deps for openmrs-api:

[INFO] +- org.openmrs.api:openmrs-api:jar:1.9.7:provided
[INFO] |  +- log4j:log4j:jar:1.2.15:runtime
[INFO] |  +- stax:stax-api:jar:1.0.1:compile
[INFO] |  +- dom4j:dom4j:jar:1.6.1:compile
[INFO] |  +- xml-apis:xml-apis:jar:1.3.03:compile

Incredibly enough, maven 3.2.1 will NOT bring log4j as runtime, it will be a provided scope (I guess that bug was fixed), so I didn't took the time to look at it.
Both dom4j and stax come from poi-ooxml, so no big deal (I will come back to these guys later).

Here be dragons.
xml-apis is the only one causing trouble here. But my gut feeling is that maven is just working as designed. I might be wrong, I guess I should raise an issue and let maven people decide it by themselves.
I will try to explain it below, but I attached an example (better to play with it).

Reporting omod module is including two relevant dependencies:
       <dependency>
            <groupId>org.openmrs.api</groupId>
            <artifactId>openmrs-api</artifactId>
            <version>${openMRSVersion}</version>
            <type>jar</type>
            <scope>provided</scope>
        </dependency>
And:
         <dependency>
            <groupId>org.apache.poi</groupId>
            <artifactId>poi-ooxml</artifactId>
            <version>3.9</version>
            <exclusions>
                <exclusion>
                    <groupId>xml-apis</groupId>
                    <artifactId>xml-apis</artifactId>
                </exclusion>
            </exclusions>
        </dependency>

 

So, maven is trying to parse the dependencies. It's looking at 'openmrs-api', and it finds a "dom4j". Yeah, it was supposed to be provided (the same as openmrs-api), but hold on! You added 'poi-ooxml' as a compile dependency, which will bring 'dom4j' as a compile dependency. So, maven decide it's a compile dependency when analysing 'openmrs-api'. Turns out that other 'openmrs-api', 'xml-apis' is a compile dependency of 'dom4j', so YEY, let's change it to compile as well. Look, you excluded the 'xml-apis' for 'poi-ooxml', but not when including 'dom4j' which became a compile dependency of 'openmrs-api'.

The sane thing I guess it to do is exclude 'dom4j' from 'poi-ooxml'. I guess it should not break anything, as you can use the provided one, right?

To do some tests using the example attached, download the tar.gz, unzip it, and go to the example-excluding-jar (it is like the reporting omod). The other project, example-provided-jar, mimics openmrs-api.
Feel free to remove the 'dom4j' exclusion to reproduce the same behaviour you have currently. 

To run it:
$ (cd ../example-provided-jar/; mvn3 clean install);  ~mvn3 org.apache.maven.plugins:maven-dependency-plugin:2.8:tree

I hope that answers the question.

Cheers,

[Michael Seaton]

Hi Cintia,

This is a great help, thank you!!

I would still consider this a bug in Maven I think, but as long as there is a way to work around it I'm happy enough.

So, I removed the xml-api exclusion that I had added to the openmrs-api dependencies as a workaround, and instead excuded dom4j from poi-ooxml as you suggested and this worked!  log4j was still included, so I upgraded to the latest version of Maven as you suggested (3.2.1), and voila, this is now gone too!

I'm "happy" with this solution, though am worried it can and will crop up again for me and others.  It's not like most of us are in the habit of inspecting the lib folder of our built omods regularly :)

For those running CI builds of reporting (OpenMRS, Mirebalais, etc), I'd recommend that we upgrade the version of Maven on those servers to 3.2.1...

Thanks again for your help with this!

Cheers,
Mike

[Cintia Del Rio]

So, the issue I raised :)

https://jira.codehaus.org/browse/MNG-5622

Let's see what the devs answer.

> For those running CI builds of reporting (OpenMRS, Mirebalais, etc), I'd recommend that we upgrade the version of Maven on those servers to 3.2.1...

Yes, sounds just about right. I think I tested a few days ago, and the default maven 3 was 3.2.1. Unfortunately, I cannot test it now because I cannot login into Bamboo
(╯°□°）╯︵ ┻━┻

>I'm "happy" with this solution, though am worried it can and will crop up again for me and others.  It's not like most of us are in the habit of inspecting the lib folder of our built omods regularly :)

Look, I guess this is the first time it bite us, right? So, let's say it becomes a big problem, we could automate something* to find out when a module is including a already-provided dependency:

• We could create a task (a bash script or similar) on each relevant module, unzip it and test the jars against openmrs-core war
• We could improve/create a new goal to the maven plugin (which will do a similar thing), add a "validation" to every module.
  
• We could try to change the package goal of the omod plugin to exclude some subtrees. I wonder if we can do it using maven3 resolution. 

(* I don't believe in manual tests for repetitive tasks :P)

The tricky part is: which openmrs-api/core should we consider when testing? The latest released? The development one? The one used in the module's pom file? Hum.

Anyway, all of the above requires some work (and more code means more maintenance). It doesn't feel like it's worth doing something about it right now (we can rant a lot about the inability of maven dealing nicely with bundle/OSGI. They are second class citizens).

As a general rule, if you are developing a module and adding a compile/runtime dependency, you need to exclude all dependencies already provided from openmrs-core (no news here ;) ).
Do we recommend developers to look into the openmrs-cor war lib folder or what? I guess getting that list/deploying to maven/updating some documentation should be easy and good enough.


Cheers,

On 20 April 2014 00:11, Michael Seaton <mse...@pih.org> wrote:

Hi Cintia,

This is a great help, thank you!!

I would still consider this a bug in Maven I think, but as long as there is a way to work around it I'm happy enough.

So, I removed the xml-api exclusion that I had added to the openmrs-api dependencies as a workaround, and instead excuded dom4j from poi-ooxml as you suggested and this worked!  log4j was still included, so I upgraded to the latest version of Maven as you suggested (3.2.1), and voila, this is now gone too!

I'm "happy" with this solution, though am worried it can and will crop up again for me and others.  It's not like most of us are in the habit of inspecting the lib folder of our built omods regularly :)

For those running CI builds of reporting (OpenMRS, Mirebalais, etc), I'd recommend that we upgrade the version of Maven on those servers to 3.2.1...

Thanks again for your help with this!

Cheers,
Mike