Clojure 1.10 "Illegal Reflective Access Operation"

[Alan Thompson]

Upgrading from Clojure 1.9 to 1.10, I am getting a new warning:

WARNING: An illegal reflective access operation has occurred

WARNING: Illegal reflective access by clojure.lang.InjectedInvoker/0x0000000800231c40 (file:/home/alan/.m2/repository/org/clojure/clojure/1.10.0/clojure-1.10.0.jar) to method com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl.parse(org.xml.sax.InputSource,org.xml.sax.HandlerBase)

WARNING: Please consider reporting this to the maintainers of clojure.lang.InjectedInvoker/0x0000000800231c40

WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations

WARNING: All illegal access operations will be denied in a future release

Using Java 11 on Ubuntu 16.04.  Should I file an issue for this?

Alan

[Andy Fingerhut]

I believe this FAQ entry covers what is known about this issue, which many others have also seen: https://clojure.org/guides/faq#illegal_access

Andy

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Alan Thompson]

I was deceived by the error message.  Since I had only changed the Clojure version 1.9 => 1.10, and since the warning mentioned only Clojure code:

WARNING: Illegal reflective access by clojure.lang.InjectedInvoker/0x0000000800231c40 

   (file:/home/alan/.m2/repository/org/clojure/clojure/1.10.0/clojure-1.10.0.jar)

I thought the problem was within the Clojure source.  After investigating further, the problem was indeed a lack of type-hinting in a dependent library (Enlive).  The fix was to add 2 type hints to a function call:

   (.parse

     ^java.io.InputStream                s     ; actual type => java.io.BufferedInputStream

     ^org.xml.sax.helpers.DefaultHandler ch    ; actual type => net.cgrand.xml.proxy$org.xml.sax.ext.DefaultHandler2

   )

A Pull Request has been filed.  Much thanks to Andy Fingerhut for pointing me in the right direction.

Alan Thompson

[Matching Socks]

Does this need adjusting in clojure.xml too?  The code looks pretty similar:

(defn startparse-sax [s ch]
  (.. SAXParserFactory (newInstance) (newSAXParser) (parse s ch)))

The reflection on "parse" is convenient. There are multiple SaxParser.parse methods with unique capabilities. The String method allows you not to know how the data is encoded.  To open an InputStream, you have to know the encoding.  It's pretty hard to open an XML instance correctly!  And the InputSource method allows you to parse from a Reader, e.g., a StringReader.

[Alan Thompson]

After further investigation, I modified my copy of the xml parsing code to wrap either an InputStream or a Reader with an org.xml.sax.InputSource (and changed the type hint):

(defn ^:private sax-parse-fn

  [xml-input content-handler]

  (let [input-source (cond

                       (or (instance? InputStream xml-input)

                           (instance? Reader xml-input))              (org.xml.sax.InputSource. xml-input)

                       (instance? org.xml.sax.InputSource xml-input)  xml-input

                       :else (throw (ex-info "sax-parse-fn: xml-input must be one of InputStream, Reader, or org.xml.sax.InputSource"

                                      {:type  (type xml-input)

                                       :class (class xml-input)})))]

    (it-> (SAXParserFactory/newInstance)

      (doto it

        (.setValidating false)

        (.setFeature "http://xml.org/sax/features/external-general-entities" false)

        (.setFeature "http://xml.org/sax/features/external-parameter-entities" false))

      (.newSAXParser it)

      (doto it

        (.setProperty "http://xml.org/sax/properties/lexical-handler" content-handler))

      (.parse it

        ^org.xml.sax.InputSource             input-source

        ^org.xml.sax.helpers.DefaultHandler  content-handler))))

(s/defn parse       ; #todo fix docstring

  ([xml-input] (parse xml-input sax-parse-fn))

  ([xml-input parse-fn]

    (let [result-atom     (atom (xml-zip {:type :document :content nil}))

          content-handler (handler result-atom)]

      (parse-fn xml-input content-handler)

      ; #todo document logic vvv using xkcd & plain xml example

      (let [parsed-data (it-> @result-atom

                          (first it)

                          (:content it)

                          (drop-if #(= :dtd (:type %)) it)

                          (drop-if #(string? %) it)

                          (only it))]

        parsed-data))))

--

[Vitex Software]

Todays warning:

(defn fix-fields
  "Only Fields branch"
  []
  (:content (-> (clojure.zip/xml-zip (xml/parse "specs/RHUB_v2.8_QuickFIX.xml"))
                zip/down
                zip/right
                zip/right
                zip/right
                zip/right
                zip/node))
  )

(defn fix-fields->format [rec]
  {:tag     (-> rec :attrs :number Long/parseLong)
   :name    (-> rec :attrs :name)
   :type    (-> rec :attrs :type)
   :keyword (csk/->kebab-case (-> rec :attrs :name))
   :values  (-> rec :content :enum)
   })


(def fix-fields-reindexed (->> (fix-fields) (map fix-fields->format)))

=>

ARNING: An illegal reflective access operation has occurred

WARNING: Illegal reflective access by clojure.lang.InjectedInvoker/0x00007f3ca00b4c40 (file:/home/vitex/.m2/repository/org/clojure/clojure/1.10.1/clojure-1.10.1.jar) to method com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl.parse(org.xml.sax.InputSource,org.xml.sax.HandlerBase)
WARNING: Please consider reporting this to the maintainers of clojure.lang.InjectedInvoker/0x00007f3ca00b4c40

[Alex Miller]

I believe this has already been discussed in this same thread, but to rehash.

More info on what this warning means: https://clojure.org/guides/faq#illegal_access

To diagnose the cause, you can use --illegal-access=debug to get better info about the cause. Here I assume it's in xml/parse, but you haven't provided enough info to know what exactly xml is aliasing here. One likely candidate is clojure.xml and indeed clojure.xml/parse is reflective by design. You should consider that function deprecated - it does not present a full range of xml parsing options in the modern world and you should use the contrib lib org.clojure/data.xml instead (which does not have this issue).

[Alan Thompson]

I have included easy-to-use parsers of 3 types in the Tupelo library for 3 formats:

- HTML:  tupelo.parse.tagsoup

- XML:   tupelo.parse.xml

- YAML:  tupelo.parse.yaml

Enjoy!

Alan

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/clojure/cd43204d-cbd2-488e-95dc-26a392f991bc%40googlegroups.com.

[Matching Socks]

Alan, consider JSoup too, if you can... Enlive offers both options, and, if memory serves, JSoup seemed the more capable with HTML5.