SNAPSHOT artifacts resolution issue
Julien CARSIQUE
I don't know if it's linked to http://issues.jenkins-ci.org/browse/JENKINS-8670 so waiting for your analysis, I didn't
fill an issue.
Here is the use case: we are working with multiple Maven repositories (internal and public). Public repository contains
nightly built artifacts, deployed once they passed through a battery of unit and functional tests; whereas internal
artifacts are continuously build and deployed after every commit; so, internal artifacts should (almost) always take
precedence on public ones.
The issue encountered with Jenkins seems to be a regression and was already encountered by us with Maven and Nexus. It's
about the SNAPSHOT resolution.
You may have a look at the symptoms on http://qa.nuxeo.org/jenkins/job/nuxeo-core-5.4/368/console (a
dependencyManagement entry for nuxeo-runtime-remoting was added in nuxeo-runtime-parent module, which POM has been
deployed in our internal Maven repository but not yet in our public Maven repository):
Parsing POMs
downloaded artifact
http://maven.in.nuxeo.com/nexus/content/repositories/public-snapshots//org/nuxeo/runtime/nuxeo-runtime-parent/5.4.1-SNAPSHOT/maven-metadata.xml
downloaded artifact
http://maven.in.nuxeo.com/nexus/content/groups/internal-snapshots//org/nuxeo/runtime/nuxeo-runtime-parent/5.4.1-SNAPSHOT/maven-metadata.xml
(...)
ERROR: Failed to parse POMs
(...)
Caused by: hudson.remoting.ProxyException: hudson.maven.MavenModuleSetBuild$MavenExecutionException:
org.apache.maven.project.ProjectBuildingException: Some problems were encountered while processing the POMs:
[ERROR] 'dependencies.dependency.version' for org.nuxeo.runtime:nuxeo-runtime-remoting:jar is missing. @ line 23, column 17
at hudson.maven.MavenModuleSetBuild$PomParser.invoke(MavenModuleSetBuild.java:1198)
at hudson.maven.MavenModuleSetBuild$PomParser.invoke(MavenModuleSetBuild.java:991)
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:1931)
at hudson.remoting.UserRequest.perform(UserRequest.java:114)
at hudson.remoting.UserRequest.perform(UserRequest.java:48)
at hudson.remoting.Request$2.run(Request.java:270)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: hudson.remoting.ProxyException: org.apache.maven.project.ProjectBuildingException: Some problems were
encountered while processing the POMs:
[ERROR] 'dependencies.dependency.version' for org.nuxeo.runtime:nuxeo-runtime-remoting:jar is missing. @ line 23, column 17
at org.apache.maven.project.DefaultProjectBuilder.build(DefaultProjectBuilder.java:325)
at hudson.maven.MavenEmbedder.buildProjects(MavenEmbedder.java:360)
at hudson.maven.MavenEmbedder.readProjects(MavenEmbedder.java:330)
at hudson.maven.MavenEmbedder.readProject(MavenEmbedder.java:321)
at hudson.maven.MavenModuleSetBuild$PomParser.readChilds(MavenModuleSetBuild.java:1253)
at hudson.maven.MavenModuleSetBuild$PomParser.invoke(MavenModuleSetBuild.java:1159)
... 11 more
Because you won't have access to our internal repositories, here are the contents of public and internal metadata files.
http://maven.in.nuxeo.com/nexus/content/repositories/public-snapshots/org/nuxeo/runtime/nuxeo-runtime-parent/5.4.1-SNAPSHOT/maven-metadata.xml
<metadata>
<groupId>org.nuxeo.runtime</groupId>
<artifactId>nuxeo-runtime-parent</artifactId>
<version>5.4.1-SNAPSHOT</version>
<versioning>
<snapshot>
<timestamp>20110202.045408</timestamp>
<buildNumber>48</buildNumber>
</snapshot>
<lastUpdated>20110202045420</lastUpdated>
</versioning>
</metadata>
http://maven.in.nuxeo.com/nexus/content/groups/internal-snapshots/org/nuxeo/runtime/nuxeo-runtime-parent/5.4.1-SNAPSHOT/maven-metadata.xml
<metadata>
<groupId>org.nuxeo.runtime</groupId>
<artifactId>nuxeo-runtime-parent</artifactId>
<version>5.4.1-SNAPSHOT</version>
<versioning>
<snapshot>
<timestamp>20110206.141840</timestamp>
<buildNumber>18</buildNumber>
</snapshot>
<lastUpdated>20110206141840</lastUpdated>
</versioning>
</metadata>
As you can see that the "internal" snapshot artifact with timestamp 20110206.141840 is more recent than the public one
with timestamp 20110202.045408.
But when parsing the POM, comparing metadata, Jenkins choses to download the bad artifact, probably because of the
"buildNumber" which is... simply useless and meaningless. That was an issue in Maven 2, still unfixed I guess in Maven
3, for which Nexus developed a workaround if I'm not wrong.
Since Jenkins is parsing POM files, it has to also workaround with this issue, if it still unfixed in Maven.
This is the "visible part of the iceberg", raising an issue and blocking the build. But it could lead to some issues
more difficult to analyze in case it doesn't fail the build but assemble deprecated code instead of new one.
Thanks for help,
--
Julien Carsique, DevOps, Nuxeo (Paris, France)
www.nuxeo.com - The Open Source ECM Platform - www.nuxeo.org
Nuxeo ECM Stack - The Java EE, scalable, standard-based ECM Platform
Olivier Lamy
How do you deploy your artifacts ?
With : mvn ... deploy ... (as goals in your job)
Or with the maven deployer publisher plugin from jenkins
--
Olivier Lamy
http://twitter.com/olamy
http://www.linkedin.com/in/olamy
2011/2/9 Julien CARSIQUE <jcar...@nuxeo.com>:
Julien CARSIQUE
Not sure why and in which cases I should use the maven deployer publisher plugin (post build action configured with
checkbox "deploy artifacts..." ?). Last time I tried this, it was not working, but I don't remember why (missing
profile, ...).
Looking at the job configuration, there are some new options I don't use:
"Resolve Dependencies during Pom parsing", "Process Plugins during Pom parsing", "Maven Validation Level" (set to
LEVEL_MINIMAL).
I don't understand why there's a POM parsing whereas the option is unchecked in the job configuration:
Found mavenVersion 2.2.1 from file jar:file:/opt/apache-maven-2.2.1/lib/maven-2.2.1-uber.jar!/META-INF/maven/org.apache.maven/maven-core/pom.properties
Parsing POMs
downloaded artifact http://maven.in.nuxeo.com/nexus/content/repositories/public-snapshots//org/nuxeo/runtime/nuxeo-runtime-parent/5.4.1-SNAPSHOT/maven-metadata.xml
downloaded artifact http://maven.in.nuxeo.com/nexus/content/groups/internal-snapshots//org/nuxeo/runtime/nuxeo-runtime-parent/5.4.1-SNAPSHOT/maven-metadata.xml
downloaded artifact http://maven.in.nuxeo.com/nexus/content/repositories/public-snapshots//org/nuxeo/nuxeo-ecm/5.4.1-SNAPSHOT/maven-metadata.xml
downloaded artifact http://maven.in.nuxeo.com/nexus/content/groups/internal-snapshots//org/nuxeo/nuxeo-ecm/5.4.1-SNAPSHOT/maven-metadata.xml
failed to transfer Unable to locate resource in repository
downloaded artifact http://maven.in.nuxeo.com/nexus/content/repositories/public-snapshots//org/nuxeo/runtime/nuxeo-runtime-parent/5.4.1-SNAPSHOT/maven-metadata.xml
downloaded artifact http://maven.in.nuxeo.com/nexus/content/groups/internal-snapshots//org/nuxeo/runtime/nuxeo-runtime-parent/5.4.1-SNAPSHOT/maven-metadata.xml
failed to transfer Unable to locate resource in repository
downloaded artifact http://maven.in.nuxeo.com/nexus/content/repositories/public-snapshots//org/nuxeo/nuxeo-ecm/5.4.1-SNAPSHOT/maven-metadata.xml
downloaded artifact http://maven.in.nuxeo.com/nexus/content/groups/internal-snapshots//org/nuxeo/nuxeo-ecm/5.4.1-SNAPSHOT/maven-metadata.xml
downloaded artifact http://maven.in.nuxeo.com/nexus/content/repositories/public-snapshots//org/nuxeo/runtime/nuxeo-runtime-parent/5.4.1-SNAPSHOT/maven-metadata.xml
downloaded artifact http://maven.in.nuxeo.com/nexus/content/groups/internal-snapshots//org/nuxeo/runtime/nuxeo-runtime-parent/5.4.1-SNAPSHOT/maven-metadata.xml
downloaded artifact http://maven.in.nuxeo.com/nexus/content/repositories/public-snapshots//org/nuxeo/nuxeo-ecm/5.4.1-SNAPSHOT/maven-metadata.xml
downloaded artifact http://maven.in.nuxeo.com/nexus/content/groups/internal-snapshots//org/nuxeo/nuxeo-ecm/5.4.1-SNAPSHOT/maven-metadata.xml
[nuxeo-core-5.4] $ /usr/lib/jvm/java-6-sun//bin/java -Xmx1024m -Xms256m -XX:MaxPermSize=512m -cp /home/hudson/tmp/maven-agent.jar:/home/hudson/tmp/classworlds.jar hudson.maven.agent.Main /opt/apache-maven-2.2.1 /opt/hudson/bin/slave-1.396.jar /home/hudson/tmp/maven-interceptor.jar 50648 /home/hudson/tmp/maven2.1-interceptor.jar
<===[HUDSON REMOTING CAPACITY]===>���channel started
Executing Maven: -B -f /home/hudson/tmp/workspace/nuxeo-core-5.4/pom.xml pmd:cpd clean deploy -Pqa
[INFO] Scanning for projects...
Le 10/02/11 11:05, Olivier Lamy a écrit :
> Salut,
>
> How do you deploy your artifacts ?
> With : mvn ... deploy ... (as goals in your job)
> Or with the maven deployer publisher plugin from jenkins
>
--
Olivier Lamy
Is this repo http://maven.in.nuxeo.com/nexus/content/groups/internal-snapshots
a proxy ?
Maybe there is Metadata Max Age to configure ?
Are you using -U ?
2011/2/10 Julien CARSIQUE <jcar...@nuxeo.com>:
--
Julien CARSIQUE
I don't know why I never had this issue with Hudson/Jenkins before. I had it only within maven executions and applied
some ugly workarounds.
It's not a proxy but a group.
The issue comes from the fact that Maven finds the artifact in two different repositories (proxy, group or host is not
important here) and is not able to chose the correct one. It's a Maven issue (I should have filled a Jira issue instead
of only discussing about it on Maven mailing list; it was in Oct, 2009, and appears still unfixed).
Sadly, it's impossible to deactivate a repository, so I will always have two repositories providing the same artifact.
And so, the Nexus workaround (https://issues.sonatype.org/browse/NEXUS-2472) applying on *one* repository group cannot
help me.
I'm not using -U; it doesn't change anything in that case (we've set the snapshot update policy to "always").
The Maven issue comes from
http://svn.apache.org/viewvc/maven/maven-2/branches/maven-2.2.x/maven-artifact/src/main/java/org/apache/maven/artifact/versioning/DefaultArtifactVersion.java?view=markup
where only the buildNumber is compared instead of the timestamp value.
Isn't it possible to fix that in Jenkins, as Nexus did ?
Le 10/02/11 17:05, Olivier Lamy a écrit :
jcar...@nuxeo.com | Tel: +33 1 40 33 79 87 / +33 6 09 06 38 03
Olivier Lamy
> Are you sure the pom parsing has always used the same unchanged code ?
No :-) but I'm sure the code has changed since the maven 3 integration.
The pom parsing now use maven3 api (even if your build use maven 2.x).
> I don't know why I never had this issue with Hudson/Jenkins before. I had it only within maven executions and applied
> some ugly workarounds.
>
> It's not a proxy but a group.
> The issue comes from the fact that Maven finds the artifact in two different repositories (proxy, group or host is not
> important here) and is not able to chose the correct one. It's a Maven issue (I should have filled a Jira issue instead
> of only discussing about it on Maven mailing list; it was in Oct, 2009, and appears still unfixed).
> Sadly, it's impossible to deactivate a repository, so I will always have two repositories providing the same artifact.
> And so, the Nexus workaround (https://issues.sonatype.org/browse/NEXUS-2472) applying on *one* repository group cannot
> help me.
>
> I'm not using -U; it doesn't change anything in that case (we've set the snapshot update policy to "always").
>
> The Maven issue comes from
> http://svn.apache.org/viewvc/maven/maven-2/branches/maven-2.2.x/maven-artifact/src/main/java/org/apache/maven/artifact/versioning/DefaultArtifactVersion.java?view=markup
> where only the buildNumber is compared instead of the timestamp value.
>
> Isn't it possible to fix that in Jenkins, as Nexus did ?
As we are using maven3 apis. I'm not sure it will be possible to add a
workaroud.
--
Julien CARSIQUE
Thanks,
Le 10/02/11 22:19, Olivier Lamy a écrit :