Joda Time version conflict? "Initializing JollyDayHoliday for SUTime"

[gingers...@gmail.com]

I think I have a conflict involving different libraries using different versions of JodaTime. I have no idea how to fix this.

I have nearly the same problem as this:

http://stackoverflow.com/questions/21487476/maven-build-throws-jodatime-exception-at-runtime

However, in my case, I'm building a web app in Clojure, whereas my co-worker is building our Natural Language Processing engine in Java. The NLP is included as a jar. We had this working for a few weeks, but my co-worker has added some new dependencies that are now giving us this error.

This person also reports a similar error:

 Error creating edu.stanford.nlp.time.TimeExpressionExtractorImpl

See here:

http://mail.wso2.org/mailarchive/dev/2014-September/035337.html

with an answer here:

http://mail.wso2.org/mailarchive/dev/2014-September/035341.html

The issue in both cases seems to be a version conflict in different libraries, but I have no idea how to resolve this. In the Clojure app we use clj-time, which apparently uses Joda-Time 2.6, whereas the Stanford Core NLP libraries seem to use Joda 2.1. Is there anyway to resolve that conflict?

Both pieces of software compile, and the Clojure app can start, with the NLP engine included. However, the Clojure app then calls a "start" method in the NLP engine, which causes the NLP engine to load the lexers and parsers that it needs. We then get these messages and errors:

Loading classifier from /home/safflower/apricots/dependencies/english.muc.7class.caseless.distsim.crf.ser.gz ... done [1.5 sec].

Loading parser from serialized file edu/stanford/nlp/models/lexparser/englishPCFG.ser.gz ...  done [0.3 sec].

Loading parser from serialized file edu/stanford/nlp/models/lexparser/englishPCFG.caseless.ser.gz ...  done [1.1 sec].

Adding annotator tokenize

TokenizerAnnotator: No tokenizer type provided. Defaulting to PTBTokenizer.

Adding annotator ssplit

Adding annotator pos

Reading POS tagger model from edu/stanford/nlp/models/pos-tagger/english-left3words/english-left3words-distsim.tagger ... done [1.9 sec].

Adding annotator lemma

Adding annotator ner

Loading classifier from /home/safflower/apricots/dependencies/english.all.7class.distsim.crf.ser.gz ... done [1.5 sec].

Initializing JollyDayHoliday for SUTime from classpath: edu/stanford/nlp/models/sutime/jollyday/Holidays_sutime.xml as sutime.binder.1.
#<ReflectionLoadingException edu.stanford.nlp.util.ReflectionLoading$ReflectionLoadingException: Error creating edu.stanford.nlp.time.TimeExpressionExtractorImpl>

Exception in start/start: edu.stanford.nlp.util.ReflectionLoading$ReflectionLoadingException: Error creating edu.stanford.nlp.time.TimeExpressionExtractorImpl    

If I do this in the Java app:

mvn dependency:tree

I see this:

[INFO] |  +- joda-time:joda-time:jar:2.1:compile
[INFO] |  \- javax.xml.bind:jaxb-api:jar:2.2.7:compile
[INFO] +- log4j:log4j:jar:1.2.17:compile
[INFO] +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:compile
[INFO] |  \- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile
[INFO] +- junit:junit:jar:3.8.1:test
[INFO] +- net.sf.supercsv:super-csv:jar:2.0.0-beta-1:compile
[INFO] +- org.apache.commons:commons-lang3:jar:3.0:compile
[INFO] \- edu.stanford.nlp:stanford-corenlp:jar:3.5.2:compile
[INFO]    +- com.io7m.xom:xom:jar:1.2.10:compile
[INFO]    |  +- xml-apis:xml-apis:jar:1.3.03:compile
[INFO]    |  +- xerces:xercesImpl:jar:2.8.0:compile
[INFO]    |  \- xalan:xalan:jar:2.7.0:compile
[INFO]    +- com.googlecode.efficient-java-matrix-library:ejml:jar:0.23:compile
[INFO]    \- javax.json:javax.json-api:jar:1.0:compile

If I do this in the Clojure app:

lein deps :tree

I see this:

Retrieving org/clojure/tools.nrepl/0.2.6/tools.nrepl-0.2.6.pom from central
Retrieving clojure-complete/clojure-complete/0.2.3/clojure-complete-0.2.3.pom from clojars
Retrieving org/clojure/tools.nrepl/0.2.6/tools.nrepl-0.2.6.jar from central
Retrieving clojure-complete/clojure-complete/0.2.3/clojure-complete-0.2.3.jar from clojars
Possibly confusing dependencies found:
[slingshot "0.10.3"]
 overrides
[clj-http "1.1.2"] -> [slingshot "0.12.2" :exclusions [org.clojure/clojure]]

Consider using these exclusions:
[clj-http "1.1.2" :exclusions [slingshot]]

[clj-time "0.6.0"]
 overrides
[ring "1.4.0-RC1"] -> [ring/ring-jetty-adapter "1.4.0-RC1"] -> [ring/ring-core "1.4.0-RC1"] -> [clj-time "0.9.0"]
 and
[ring "1.4.0-RC1"] -> [ring/ring-devel "1.4.0-RC1"] -> [ring/ring-core "1.4.0-RC1"] -> [clj-time "0.9.0"]
 and
[ring "1.4.0-RC1"] -> [ring/ring-core "1.4.0-RC1"] -> [clj-time "0.9.0"]

Consider using these exclusions:
[ring "1.4.0-RC1" :exclusions [clj-time]]
[ring "1.4.0-RC1" :exclusions [clj-time]]
[ring "1.4.0-RC1" :exclusions [clj-time]]

[org.clojure/tools.namespace "0.2.4"]
 overrides
[ring "1.4.0-RC1"] -> [ring/ring-devel "1.4.0-RC1"] -> [ns-tracker "0.3.0"] -> [org.clojure/tools.namespace "0.2.10"]

Consider using these exclusions:
[ring "1.4.0-RC1" :exclusions [org.clojure/tools.namespace]]

[clj-stacktrace "0.2.7"]
 overrides
[ring "1.4.0-RC1"] -> [ring/ring-devel "1.4.0-RC1"] -> [clj-stacktrace "0.2.8"]

Consider using these exclusions:
[ring "1.4.0-RC1" :exclusions [clj-stacktrace]]

[clj-time "0.6.0"] -> [joda-time "2.2"]
 overrides
[ring "1.4.0-RC1"] -> [ring/ring-jetty-adapter "1.4.0-RC1"] -> [ring/ring-core "1.4.0-RC1"] -> [clj-time "0.9.0"] -> [joda-time "2.6"]
 and
[ring "1.4.0-RC1"] -> [ring/ring-devel "1.4.0-RC1"] -> [ring/ring-core "1.4.0-RC1"] -> [clj-time "0.9.0"] -> [joda-time "2.6"]
 and
[ring "1.4.0-RC1"] -> [ring/ring-core "1.4.0-RC1"] -> [clj-time "0.9.0"] -> [joda-time "2.6"]

Consider using these exclusions:
[ring "1.4.0-RC1" :exclusions [joda-time]]
[ring "1.4.0-RC1" :exclusions [joda-time]]
[ring "1.4.0-RC1" :exclusions [joda-time]]

 [cheshire "5.5.0"]
   [com.fasterxml.jackson.core/jackson-core "2.5.3"]
   [com.fasterxml.jackson.dataformat/jackson-dataformat-cbor "2.5.3"]
   [com.fasterxml.jackson.dataformat/jackson-dataformat-smile "2.5.3"]
   [tigris "0.1.1"]
 [clj-http "1.1.2"]
   [com.cognitect/transit-clj "0.8.271" :exclusions [[org.clojure/clojure]]]
     [com.cognitect/transit-java "0.8.287"]
       [com.fasterxml.jackson.datatype/jackson-datatype-json-org "2.3.2"]
         [com.fasterxml.jackson.core/jackson-databind "2.3.2"]
           [com.fasterxml.jackson.core/jackson-annotations "2.3.0"]
         [org.json/json "20090211"]
       [org.apache.directory.studio/org.apache.commons.codec "1.8"]
       [org.msgpack/msgpack "0.6.10"]
         [com.googlecode.json-simple/json-simple "1.1.1" :exclusions [[junit]]]
         [org.javassist/javassist "3.18.1-GA"]
   [commons-codec "1.10" :exclusions [[org.clojure/clojure]]]
   [commons-io "2.4" :exclusions [[org.clojure/clojure]]]
   [crouton "0.1.2" :exclusions [[org.clojure/clojure]]]
     [org.jsoup/jsoup "1.7.1"]
   [org.apache.httpcomponents/httpclient "4.4.1" :exclusions [[org.clojure/clojure]]]
     [commons-logging "1.2"]
   [org.apache.httpcomponents/httpcore "4.4.1" :exclusions [[org.clojure/clojure]]]
   [org.apache.httpcomponents/httpmime "4.4.1" :exclusions [[org.clojure/clojure]]]
   [org.clojure/tools.reader "0.9.2" :exclusions [[org.clojure/clojure]]]
   [potemkin "0.3.13" :exclusions [[org.clojure/clojure]]]
     [clj-tuple "0.2.1"]
 [clj-stacktrace "0.2.7"]
 [clj-time "0.6.0"]
   [joda-time "2.2"]
 [clojure-complete "0.2.3" :scope "test" :exclusions [[org.clojure/clojure]]]
 [com.novemberain/monger "2.0.1"]
   [clojurewerkz/support "1.1.0"]
     [com.google.guava/guava "18.0"]
   [org.mongodb/mongo-java-driver "2.12.4"]
 [com.taoensso/timbre "3.2.1"]
   [com.taoensso/encore "1.5.1"]
   [io.aviso/pretty "0.1.10"]
 [compojure "1.3.4"]
   [clout "2.1.2"]
     [instaparse "1.4.0" :exclusions [[org.clojure/clojure]]]
   [medley "0.6.0"]
   [org.clojure/tools.macro "0.1.5"]
   [ring/ring-codec "1.0.0"]
 [dire "0.5.1"]
   [robert/hooke "1.3.0"]
 [local/nlp "1.0-SNAPSHOT"]
 [manifold "0.1.0"]
   [org.clojure/tools.logging "0.3.1"]
   [riddley "0.1.9"]
 [me.raynes/fs "1.4.4"]
   [org.apache.commons/commons-compress "1.4"]
     [org.tukaani/xz "1.0"]
 [org.clojure/clojure "1.6.0"]
 [org.clojure/core.cache "0.6.4"]
   [org.clojure/data.priority-map "0.0.4"]
 [org.clojure/core.incubator "0.1.3"]
 [org.clojure/core.match "0.3.0-alpha4"]
   [org.clojure/tools.analyzer.jvm "0.6.5"]
     [org.clojure/core.memoize "0.5.6"]
     [org.clojure/tools.analyzer "0.6.4"]
     [org.ow2.asm/asm-all "4.2"]
 [org.clojure/data.json "0.2.5"]
 [org.clojure/tools.namespace "0.2.4"]
 [org.clojure/tools.nrepl "0.2.6" :scope "test" :exclusions [[org.clojure/clojure]]]
 [overtone/at-at "1.2.0"]
 [ring/ring-json "0.3.1"]
 [ring "1.4.0-RC1"]
   [ring/ring-core "1.4.0-RC1"]
     [commons-fileupload "1.3.1"]
     [crypto-equality "1.0.0"]
     [crypto-random "1.2.0"]
   [ring/ring-devel "1.4.0-RC1"]
     [hiccup "1.0.5"]
     [ns-tracker "0.3.0"]
       [org.clojure/java.classpath "0.2.2"]
   [ring/ring-jetty-adapter "1.4.0-RC1"]
     [org.eclipse.jetty/jetty-server "9.2.10.v20150310"]
       [javax.servlet/javax.servlet-api "3.1.0"]
       [org.eclipse.jetty/jetty-http "9.2.10.v20150310"]
         [org.eclipse.jetty/jetty-util "9.2.10.v20150310"]
       [org.eclipse.jetty/jetty-io "9.2.10.v20150310"]
   [ring/ring-servlet "1.4.0-RC1"]
 [slingshot "0.10.3"]


   [robert/hooke "1.3.0"]
 [local/nlp "1.0-SNAPSHOT"]
 [manifold "0.1.0"]
   [org.clojure/tools.logging "0.3.1"]
   [riddley "0.1.9"]
 [me.raynes/fs "1.4.4"]
   [org.apache.commons/commons-compress "1.4"]
     [org.tukaani/xz "1.0"]
 [org.clojure/clojure "1.6.0"]
 [org.clojure/core.cache "0.6.4"]
   [org.clojure/data.priority-map "0.0.4"]
 [org.clojure/core.incubator "0.1.3"]
 [org.clojure/core.match "0.3.0-alpha4"]
   [org.clojure/tools.analyzer.jvm "0.6.5"]
     [org.clojure/core.memoize "0.5.6"]
     [org.clojure/tools.analyzer "0.6.4"]
     [org.ow2.asm/asm-all "4.2"]
 [org.clojure/data.json "0.2.5"]
 [org.clojure/tools.namespace "0.2.4"]
 [org.clojure/tools.nrepl "0.2.6" :scope "test" :exclusions [[org.clojure/clojure]]]
 [overtone/at-at "1.2.0"]
 [ring/ring-json "0.3.1"]
 [ring "1.4.0-RC1"]
   [ring/ring-core "1.4.0-RC1"]
     [commons-fileupload "1.3.1"]
     [crypto-equality "1.0.0"]
     [crypto-random "1.2.0"]
   [ring/ring-devel "1.4.0-RC1"]
     [hiccup "1.0.5"]
     [ns-tracker "0.3.0"]
       [org.clojure/java.classpath "0.2.2"]
   [ring/ring-jetty-adapter "1.4.0-RC1"]
     [org.eclipse.jetty/jetty-server "9.2.10.v20150310"]
       [javax.servlet/javax.servlet-api "3.1.0"]
       [org.eclipse.jetty/jetty-http "9.2.10.v20150310"]
         [org.eclipse.jetty/jetty-util "9.2.10.v20150310"]
       [org.eclipse.jetty/jetty-io "9.2.10.v20150310"]
   [ring/ring-servlet "1.4.0-RC1"]
 [slingshot "0.10.3"]

How would I resolve a potential version conflict?

This is the project.clj file that I have:

(defproject oyster "0.1"
  :dependencies [[org.clojure/clojure "1.6.0"]
                 [com.taoensso/timbre "3.2.1"]
                 [dire "0.5.1"]
                 [slingshot "0.10.3"]
                 [ring "1.4.0-RC1"]
                 [clj-time "0.6.0"]
                 [org.clojure/data.json "0.2.5"]
                 [compojure "1.3.4"]
                 [com.novemberain/monger "2.0.1"]
                 [org.clojure/tools.namespace "0.2.4"]
                 [manifold "0.1.0"]  
                 [me.raynes/fs "1.4.4"]
                 [org.clojure/core.incubator "0.1.3"]
                 [clj-stacktrace "0.2.7"]
                 [overtone/at-at "1.2.0"]
                 [ring/ring-json "0.3.1"]
                 [clj-http "1.1.2"]
                 [org.clojure/core.cache "0.6.4"]
                 [cheshire "5.5.0"]
                 [org.clojure/core.match "0.3.0-alpha4"]
                 [local/nlp "1.0-SNAPSHOT"]]
  :repositories {"local" ~(str (.toURI (java.io.File. "maven_repository")))}
  :disable-implicit-clean true
;;  :warn-on-reflection true
  :source-paths      ["src/clojure"]
  :java-source-paths ["src/java"]
  :main oyster.core
  :aot :all
  :jvm-opts ["-Xms100m" "-Xmx1000m" "-XX:-UseCompressedOops"])

[Gary Verhaegen]

Assuming there is a version that works for both dependencies, you can
manually fix it in your own project.clj. Your own direct dependencies
will override transitive ones.

Otherwise, as far as I can tell, you're stuck. Maybe you can try using
an older clj-time?

> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com
> Note that posts from new members are moderated - please be patient with your
> first post.
> To unsubscribe from this group, send email to
> clojure+u...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+u...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[gingers...@gmail.com]

> Otherwise, as far as I can tell, you're stuck. Maybe you can try using
> an older clj-time?

That's an interesting idea. I see that this:

                 [clj-time "0.4.5"]

rolls back JodaTime to 2.1:

https://clojars.org/clj-time/versions/0.4.5

which is the same dependency as in the Java app. So, fantastic. But when I try and then try to compile, I get:

java.lang.IllegalAccessError: in-seconds does not exist, compiling:(cookies.clj:1:1)
        at clojure.lang.Compiler$InvokeExpr.eval(Compiler.java:3558)
        at clojure.lang.Compiler.compile1(Compiler.java:7226)
        at clojure.lang.Compiler.compile1(Compiler.java:7216)
        at clojure.lang.Compiler.compile(Compiler.java:7292)
        at clojure.lang.RT.compile(RT.java:398)
        at clojure.lang.RT.load(RT.java:438)
        at clojure.lang.RT.load(RT.java:411)
        at clojure.core$load$fn__5066.invoke(core.clj:5641)
        at clojure.core$load.doInvoke(core.clj:5640)
        at clojure.lang.RestFn.invoke(RestFn.java:408)
        at clojure.core$load_one.invoke(core.clj:5446)
        at clojure.core$load_lib$fn__5015.invoke(core.clj:5486)
        at clojure.core$load_lib.doInvoke(core.clj:5485)
        at clojure.lang.RestFn.applyTo(RestFn.java:142)
        at clojure.core$apply.invoke(core.clj:626)
        at clojure.core$load_libs.doInvoke(core.clj:5528)
        at clojure.lang.RestFn.applyTo(RestFn.java:137)
        at clojure.core$apply.invoke(core.clj:628)
        at clojure.core$use.doInvoke(core.clj:5618)
        at clojure.lang.RestFn.invoke(RestFn.java:408)
        at compojure.handler$loading__4958__auto__.invoke(handler.clj:1)

So it looks like Compojure is depending on a newer version of JodaTime.

I am not sure how to resolve the conflicts between all the pieces of code, in our software, that is using JodaTime.

[gingers...@gmail.com]

But we had this working, so I don't think the conflict is between the NLP and the web app which uses Compojure.

On Monday, June 29, 2015 at 6:38:40 PM UTC-4, Gary Verhaegen wrote:

[Fluid Dynamics]

On Monday, June 29, 2015 at 6:38:40 PM UTC-4, Gary Verhaegen wrote:

Assuming there is a version that works for both dependencies, you can
manually fix it in your own project.clj. Your own direct dependencies
will override transitive ones.

Otherwise, as far as I can tell, you're stuck. Maybe you can try using
an older clj-time?

In theory, JodaTime 2.6 should not have any API-breaking changes since 2.1, or they should have called it 3.something.

So, if you can force the NLP module to use JodaTime 2.6 it *should* work.

If that fails, it may still be possible if you can force the two versions of JodaTime to load in separate classloaders. I haven't the foggiest how that might actually be achieved. Actually I'm somewhat surprised that the Java NLP component and clj-time are not *already* loading in different classloaders, the former in the standard Java classloader and the latter in Clojure's DynamicClassLoader. Two different versions of the same Java package or Clojure namespace can *usually* coexist peacefully in different classloaders, though, modulo native dependencies or centralized filesystem stuff (e.g. a single .foo file in the user directory that they both modify, stepping on each others' toes, and they can't be overridden to look for separate files to each keep their own version).

If the worst comes to the worst, you may need to run the NLP module and the Clojure code in separate JVMs using some form of IPC to exchange data. Then deployment, startup, and shutdown become more complicated and annoying, and communication across the divide much less efficient (like kernel mode switching, IPC means carting data across address space boundaries), so ideally there'd be a clean internal boundary between the NLP-parts and the rest that has relatively little traffic crossing it. You'd then end up with a kind of NLP daemon and an application that had a dependency on it.

[gingers...@gmail.com]

> If the worst comes to the worst, you may need to run the NLP module
> and the Clojure code in separate JVMs using some form of IPC to exchange data.

That is what I'm looking at right now, though I've also been told that I absolutely must have this working by tomorrow morning, so I'm a little frustrated with the amount of work I face tonight. Also, a month ago we were doing IPC and then we gave up on that redesigned the NLP to work as a library we could embed inside of the Clojure app, but now it looks like we need to reverse that decision.

For me, it's been a good reminder that "easy Java interop" is true up to a point, but then very not true for certain kinds of ambitions.

[Sean Corfield]

Have you actually tried any of the exclusions that Leiningen suggests? For example:

[clj-time "0.6.0"]
 overrides
[ring "1.4.0-RC1"] -> [ring/ring-jetty-adapter "1.4.0-RC1"] -> [ring/ring-core "1.4.0-RC1"] -> [clj-time "0.9.0"]
 and
[ring "1.4.0-RC1"] -> [ring/ring-devel "1.4.0-RC1"] -> [ring/ring-core "1.4.0-RC1"] -> [clj-time "0.9.0"]
 and
[ring "1.4.0-RC1"] -> [ring/ring-core "1.4.0-RC1"] -> [clj-time "0.9.0"]

Consider using these exclusions:
[ring "1.4.0-RC1" :exclusions [clj-time]]
[ring "1.4.0-RC1" :exclusions [clj-time]]
[ring "1.4.0-RC1" :exclusions [clj-time]]

This tells you to that Ring is pulling in the later version of clj-time (and, hence, the Joda Time 2.6) but clj-time 0.6.0 is pulling in 2.2 (next set of suggested exclusions). The suggestion is to add the exclusions to your Ring dependency:

[ring "1.4.0-RC1" :exclusions [clj-time joda-time]]

See if that helps.

Sean

[Gary Verhaegen]

Also, your last error (about in-ns) is from an incompatible version of clj-time, not joda-time.

By the way, this has nothing to do with interop, it's straight dependency management on the jvm.

Concerning FD's solution of classloaders, you'll also jave to figure out a way to package the different versions into your JAR, which will not be trivial if both versions use the same package names (which seem to be the case).

I think your best bet at this point is to carefully read the lein deps :tree output to note all of the required versions of clj-time and jodatime, and then try to play with either excludes or version ponning in your project.clj until you find a combination that works. Also remember that your code is the most flexible: are you absolutely stuck using clj-time 0.6.0 for your own code ?

[gingers...@gmail.com]

Sean Corfield,

Thanks for this. I can go through my code and try to remove all of the uses of JodaTime in my own code, but, as that would be some work, I'd like to first confirm the diagnoses. Can you think of a way I might be able to confirm that Joda is, in fact, the problem?

---- lawrence

[Franklin M. Siler]

On Jul 6, 2015, at 1058, gingers...@gmail.com wrote:

> Sean Corfield,
>
> Thanks for this. I can go through my code and try to remove all of the uses of JodaTime in my own code, but, as that would be some work, I'd like to first confirm the diagnoses. Can you think of a way I might be able to confirm that Joda is, in fact, the problem?
>
> ---- lawrence

I’m having trouble following the thread- if you haven’t eliminated all the conflicts according to `lein deps :tree`, doing code elimination is probably premature.

Frank

[Sean Corfield]

On Jul 6, 2015, at 8:58 AM, gingers...@gmail.com wrote:
> Thanks for this. I can go through my code and try to remove all of the uses of JodaTime in my own code, but, as that would be some work, I'd like to first confirm the diagnoses. Can you think of a way I might be able to confirm that Joda is, in fact, the problem?

That’s what lein deps :tree is for — to tell you about conflicts and how to eliminate them.

Sean Corfield -- (904) 302-SEAN
An Architect's View -- http://corfield.org/

"Perfection is the enemy of the good."
-- Gustave Flaubert, French realist novelist (1821-1880)