jmxtrans and jmxmp

[Jon Archer]

Information on this combination seems thin on the ground, wondering if anyone has got this working and if so how.

I have an application that I can connect to with visualvm as follows:

$ jvisualvm --cp:p ~/.m2/repository/org/glassfish/external/opendmk_jmxremote_optional_jar/1.0-b01-ea/opendmk_jmxremote_optional_jar-1.0-b01-ea.jar --nosplash --openjmx service:jmx:jmxmp://localhost:8998

From what I can turn up I thought I'd have to:

1) set ADDITIONAL_JARS=~/.m2/repository/org/glassfish/external/opendmk_jmxremote_optional_jar/1.0-b01-ea/opendmk_jmxremote_optional_jar-1.0-b01-ea.jar

2) specify in my JSON something like: "url" : "service:jmx:jmxmp://localhost:8998"

But it seems that's not working. In the logs I see:

1:55:00.168 [ServerScheduler_Worker-1] ERROR c.googlecode.jmxtrans.jobs.ServerJob - Error in job for server: Server [host=w2, port=1099, url=service:jmx:jmxmp://localhost:8998, cronExpression=null, numQueryThreads=2]

java.net.MalformedURLException: Unsupported protocol: jmxmp

at javax.management.remote.JMXConnectorFactory.newJMXConnector(JMXConnectorFactory.java:357) ~[na:1.7.0_67]

at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:267) ~[na:1.7.0_67]

at com.googlecode.jmxtrans.connections.JmxConnectionFactory.makeObject(JmxConnectionFactory.java:19) ~[jmxtrans-all.jar:na]

        ... etc.

I don't quite know enough about JMX and JMXMP to go further. If anyone can provide suggestions I'd be grateful.

Specifically:

- is what I'm trying to do reasonable?

- is there some further straightforward debugging I can do?

Thanks in advance!

Jon

[Jon Stevens]

Can you confirm that the java process (using ps) has that jar in the classpath?

jon

--
You received this message because you are subscribed to the Google Groups "jmxtrans" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jmxtrans+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jon Archer]

Hmm I'm not sure it does. Here's what I see:

jarcher-mbp15r:jmxtrans-249 $ ./bin/jmxtrans.sh start test.json

jarcher-mbp15r:jmxtrans-249 $ ps -ef | grep jmxtrans

563245955  7772     1   0  6:40AM ttys010    0:01.31 /Library/Java/JavaVirtualMachines/jdk1.7.0_67.jdk/Contents/Home/bin/java -server -Djava.awt.headless=true -Djava.net.preferIPv4Stack=true -Djmxtrans.log.level=debug -Djmxtrans.log.dir=. -Xms512m -Xmx512m -XX:PermSize=384m -XX:MaxPermSize=384m -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.port=2101 -jar /Users/jarcher/bin/jmxtrans-249/lib/jmxtrans-all.jar -e -f jra/example.json -s 60 -c false -a /Users/jarcher/.m2/repository/org/glassfish/external/opendmk_jmxremote_optional_jar/1.0-b01-ea/opendmk_jmxremote_optional_jar-1.0-b01-ea.jar

Looking over jmxtrans.sh it does not seem to construct a -classpath or -cp argument for the java executable but rather that -a argument to jmxtrans.

I haven't (yet) rolled up my sleeves and looked at what jmxtrans does with that -a argument, but if I mangle the path to the JAR I get an exception that makes me think the intent is to jmxtrans to load it rather than supplying it as part of the -cp to java. My grasp of classloading may be faulty though :)

Exception in thread "main" java.io.FileNotFoundException: Additional jar [/Users/jarcher/.m2/repository/org/glassfish/external/opendmk_jmxremote_optional_jar/1.0-b01-ea/opendmk_jmxremote_optional_jar-1.0-b01-eax.jar] does not exist

at com.googlecode.jmxtrans.classloader.ClassLoaderEnricher.add(ClassLoaderEnricher.java:15)

at com.googlecode.jmxtrans.JmxTransformer.main(JmxTransformer.java:89)

Further suggestions welcome!

Cheers,

Jon

[Guillaume Lederrey]

Yep, your guess is right, JmxTrans enhance the classloader at runtime with the additional jars given as the "-a". I might have tried to be too smart on that one. I'll try to reproduce the problem locally to check. I'll have more info by the end of this evening.

Note that you could also try to modify the jmxtrans.sh script to add this jar to the classpath and see if it works. If it still does not work, that would be a good indication that the problem lies elsewhere.

[Jon Archer]

Thanks Guillaume.

I had a quick go at the suggestion you made re: modifying the startup script by just stuffing it in there like this:

nohup $JAVA -server -cp ~/.m2/repository/org/glassfish/external/opendmk_jmxremote_optional_jar/1.0-b01-ea/opendmk_jmxremote_optional_jar-1.0-b01-ea.jar $JAVA_OPTS $JMXTRANS_OPTS $GC_OPTS $MONITOR_OPTS $EXEC >>$LOG_FILE 2>&1 &

Launching this way (with no ADDITIONAL_JARS env var set now) yields the following:

$ jarcher-mbp15r:jmxtrans-249 $ ps -ef | grep jmxtrans

563245955  9648     1   0 11:14AM ttys010    0:02.72 /Library/Java/JavaVirtualMachines/jdk1.7.0_67.jdk/Contents/Home/bin/java -server -cp /Users/jarcher/.m2/repository/org/glassfish/external/opendmk_jmxremote_optional_jar/1.0-b01-ea/opendmk_jmxremote_optional_jar-1.0-b01-ea.jar -Djava.awt.headless=true -Djava.net.preferIPv4Stack=true -Djmxtrans.log.level=debug -Djmxtrans.log.dir=. -Xms512m -Xmx512m -XX:PermSize=384m -XX:MaxPermSize=384m -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.port=2101 -jar /Users/jarcher/bin/jmxtrans-249/lib/jmxtrans-all.jar -e -f jra/example.json -s 60 -c false

Which looks promising but, sadly, still the same exception:

java.net.MalformedURLException: Unsupported protocol: jmxmp

Thanks for your help, it's much appreciated!

Jon

[Guillaume Lederrey]

I have to admit my ignorance ... I have never worked with jmxmp before. But I'm always willing to learn.

I wrote a simple test to play with this (https://github.com/jmxtrans/jmxtrans/blob/jmxmp/test/com/googlecode/jmxtrans/connections/JmxConnectionFactoryTests.java) using :

<groupId>org.jvnet.opendmk</groupId>

<artifactId>jmxremote_optional</artifactId>

<version>1.0_01-ea</version>

as the jar. I get the same exception as you do, whether I try to add the jar to the classpath or use my ClassLoaderEnhancer ... Problem seems to be somewhere else. I'll keep looking.

[Jon Stevens]

Not promising: http://stackoverflow.com/questions/15928462/is-java-dmk-or-opendmk-still-supported-by-oracle

[Guillaume Lederrey]

This does not look so good ...

mobile : +41 76 573 32 40
skype : Guillaume.Lederrey

projects :
* http://rwanda.ledcom.ch/
* http://trock.ch/
others :
* http://kiva.org/

[Guillaume Lederrey]

After some more testing, if I put the "jmxremote_optional.jar" in the lib/ext folder of my JRE (/usr/lib/jvm/java-7-openjdk-amd64/jre/lib/ext), it suddenly works. So it seems that there is more classloader magic at work than I initially thought.

That looks like an ugly workaround, but should solve your issue in the short term. I have no idea of what the nice and clean long term solution should be like. Any idea welcomed.

Not promising: http://stackoverflow.com/questions/15928462/is-java-dmk-or-opendmk-still-supported-by-oracle

To unsubscribe from this group and stop receiving emails from it, send an email to jmxtrans+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "jmxtrans" group.

To unsubscribe from this group and stop receiving emails from it, send an email to jmxtrans+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Jon Stevens]

The ClassLoaderEnricher loads from the System classloader. That loader doesn't have the extension classloader as a parent so it isn't going to be able to load into that classpath.

https://docs.oracle.com/javase/tutorial/ext/basics/load.html
http://stackoverflow.com/questions/7255075/is-it-possible-to-get-the-extension-class-loader-object

You can either create another special classloader just for that jar or you can try to define it under -Djava.ext.dirs=   (https://docs.oracle.com/javase/tutorial/ext/basics/spi.html#run-the-client) or you can copy that jar into the lib/ext folder at startup in jmxtrans.sh (which will probably fail due to security restrictions) or you can make it part of the .deb/.rpm install.

Good times.

jon

[Jon Archer]

Thank you, I don't think I'd have thought to try that for quite a while(!) I can confirm that works for me too. 

I haven't had to had to think too hard about classloading for some time, but I'll see if I can figure out what's going on here and if I have useful findings or a fix will bring them back to you guys. 

[Guillaume Lederrey]

I'm scared by that one ...

I don't like the idea of bundling jmxremote_optional with JmxTrans, it should be optional and I doubt that many users actually need it. Modifying the ext classloader should be prevented by a reasonable security manager setting. Adding jars to the ext/lib will impact all other applications on the system, that's a big no for me.

That leaves the -Djava.ext.dirs option, but writting a good test for that one is tricky...

Seems that https://docs.oracle.com/javase/8/docs/api/java/util/ServiceLoader.html should help, but it actually does not.

I need to think a bit more before writing anything ...

[Jon Stevens]

I was just proposing the solutions in order to better understand the problem, not evaluating the best one. I agree, several of them are non-starters. =)

-Djava.ext.dirs is what I'd go with. Testing should just be that you can start up the jvm and that jmxp is registered as a protocol.

As far as bundling goes... shrug, fuck it, I'd just do it. Ease of end-user-experience is what I like to strive for. If we get reports that there is some sort of issue with it, then we can consider what to do then. Reading through the DMK source code, I doubt it'll conflict with anything though.

License is CDDL. https://opendmk.java.net/legal_notices/index.html

jon

[Guillaume Lederrey]

I've been listening to Cyrille too much, I start to be afraid of adding dependencies ;-)

I'm going to have very limited time (buying a house, first baby on the way), but I'll try to have a look ...