Is a central/shared local Maven Repo still Not safe for Jenkins master and many slaves?

[Seena Kasmai]

Greetings - 

It seems there is an outstanding maven issue "Concurrent-safe access to local Maven repository" which would advise against setting up all Jenkins servers to use a shared (e.g NFS mount) maven local repo - Can anyone confirm or share experience on how unsafe this would be?

The issue with having different repo is managing the cleaning up for each and also not optimized when running same job in different slaves, all maven artifacts have to be republished etc...(this is for a setup of 1000s of jobs across 10 slaves)

Any suggestions/advise would be helpful.

Thanks,

--Seena 

[Baptiste Mathus]

This is actually unsafe. You take the risk to corrupt the "maven local repository" (badly named, think more about it like a cache). And there's also potential issues of having a stale dependency installed in the local cache/repository (and if it's not a snapshot, it will never be downloaded again unless you delete it).

In your case, as you're likely to have many executors per slave (which btw i wouldn't recommend) the probability to corrupt the cache is even bigger.
Then, on top of that, using NFS increases the risk even more, since (your executors count) processes will access the cache concurrently.

Having a 'private' repository and cleaning it up on a regular basis is in fact simple and quite straightforward. The only downside is the disk space, but as having more disks is certainly cheaper than spending time to debug corrupted  repository issues.

Hth

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Seena Kasmai]

Thank you, point taken and thanks for correcting the terminology (local maven repo -> m2 cache)  - I wanted to get a feedback on another related question/clarification with a follow up (this might be more of a maven/nexus question) :

We use Nexus as our snapshot and release repositories; If we instruct all of our maven installations (on each slave) to download everything from nexus (i.e. like this), would Jenkins+Maven-plugin effectively now use nexus as its local maven repo (cache), or local m2. checks will always happen before going to remote nexus?

appreciate the feedback.

--Seena

[Baptiste Mathus]

Definitely yes. All our machines (Jenkins slaves, developers boxes...) are configured with the right settings to go compulsorily through our nexus server (as we have an authenticated http proxy set up, this would generally fail if you don't).
Cheers

[Christian Willman]

No, not quite.

Your internal Nexus will act as a caching layer. You won't make repeated calls to Maven central, or other external repos. But Maven will always still use a local repository to cache artifacts right on the machine performing the Maven command. And as Baptiste mentioned, this is not safe to share.

If you are using the MavenBuild type then Jenkins can create one local repo per executor. If you are using mostly FreestyleBuild type jobs, then you can actually add a flag like this to every Maven command:

-Dmaven.repo.local=${JENKINS_HOME}/${EXECUTOR_NUMBER}

to get mostly the same effect. It's a good compromise between a global local repo (impossible) and separate local repos for every job. You'll probably find that many projects share a lot of common dependencies (and transitive dependencies) so you'll have lots of cache hits once a local repo is sufficiently populated.

As far as cleanup goes, we purge the local repos every morning. Since the local repos have predictable paths, it's very easy to write a System Groovy script which iterates over jenkins.model.Jenkins#getComputers() and removes all local repos.

This mechanism allows us to waste almost no disk space and provides very good performance. We get about one ticket per quarter WRT corrupt artifacts in the local repo, which is very good.

[Stephen Connolly]

You should write up a blog post of your setup... sounds like you have a local minima set of best practices... they may not be the best practices for everyone, but there are probably quite a lot of people who could share your local minima

[Christian Willman]

Definitely coming up, Stephen. We've got one of the largest Jenkins environments I've seen, which has forced me to solve a lot of problems around high-volume Maven builds. I'll try to write something this weekend.