[play-2.2]Can't get new relic to work on heroku with play 2.2

[Otto]

Hi,

After upgrading our Play project from Play 2.1 to 2.2.0 on Heroku our NewRelic logging stopped working. We've tried all kind of variants but somehow it won't work.

The logs contain no entries for NewRelic. We're using newrelic agent 3.1.1.

JAVA_OPTS="-Xmx384m -Xss512k -XX:+UseCompressedOops -javaagent:lib/newrelic/newrelic.jar"

=> nothing happens, no NewRelic entry in log

JAVA_OPTS="-Xmx384m -Xss512k -XX:+UseCompressedOops -J-javaagent:lib/newrelic/newrelic.jar

=> nothing happens, no NewRelic entry in log

JAVA_OPTS="-javaagent:lib/newrelic/newrelic.jar -Dnewrelic.bootstrap_classpath=true"

=> Exception in thread "main" java.lang.ClassNotFoundException: com.newrelic.bootstrap.BootstrapAgent

  at java.security.AccessController.doPrivileged(Native Method)

  at java.net.URLClassLoader.findClass(URLClassLoader.java:205)

  at java.lang.ClassLoader.loadClass(ClassLoader.java:321)

  at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)

  at java.lang.ClassLoader.loadClass(ClassLoader.java:266)

  at sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:300)

  at sun.instrument.InstrumentationImpl.loadClassAndCallPremain(InstrumentationImpl.java:358)

 FATAL ERROR in native method: processing of -javaagent failed

 

Procfile

 

Procfile

 web: target/universal/stage/bin/reactor -Dhttp.port=$PORT

=> doesn't work, supposedly the script gets java_opts automatically through enviroment

Procfile

web: target/universal/stage/bin/reactor -Dhttp.port=$PORT ${JAVA_OPTS}

JAVA_OPTS:                                                  -Xmx384m -Xss512k -XX:+UseCompressedOops -J-javaagent:lib/newrelic/newrelic.jar

=> ClassNotFoundException

JAVA_OPTS: -Xmx384m -Xss512k -XX:+UseCompressedOops -J-javaagent:lib/newrelic/newrelic.jar -Dnewrelic.bootstrap_classpath=true

=> ClassNotFoundException

Thanks in advance,
Otto

[Joe Zulli]

Hi Otto. Are you using agent version 3.2.0? If so, you have to remove the -Dnewrelic.bootstrap_classpath=true part. It is apparently no longer used.

Joe

[Valtteri Pirttilä]

New Relic and Play 2.2 seem to have some trouble working together. There are some improvements in the New Relic 3.2.1 release that improve the situation. My own Play 2.1 -> 2.2 upgrade doesn't work even with New Relic 3.2.1, but it should still be worth a shot. So I recommend upgrading New Relic to the newest version.

[Otto]

We've upgraded to 3.2.1, but it still won't work. I'll file a ticket with New Relic, maybe they've got an idea.

Otto

[Iannick Brouillette]

Hi Otto,

Did you figure out this issue?

I am also having trouble making NewRelic work with play 2.2 and newrelic-agent-3.2.3.

Tried removing the "-Dnewrelic.bootstrap_classpath=true" flag, but that did not change anything.

[Otto]

Hi,

not yet, because of the holiday time we haven't had time yet to look into that. When we find something, I'll post it here.

Cheers,

Otto

[Joe Zulli]

Are you seeing the data coming in as inaccurate, or not coming in at all? We weren't seeing any data come in with 3.2.3. I created a ticket with New Relic and they found a major bug. They gave me a patched version for now, until they can get the fix into the next release. I'd open up a support ticket so you can get the patch as well and/or wait for 3.2.4.

Joe

[René Grüner Vangsgaard]

I do not see any data in New Relic with Play 2.2.1 and New Relic 3.3.1. I have not tried any newer version yet, as the addition of the New Relic jar is controlled by my hosting provider.

Are your problems gone?

[Joe Zulli]

I'm still on the patched version of 3.2.3, and honestly I'm a little scared to upgrade after all the problems I'be had with New Relic. that said, the fix they made for me should be in 3.3.1. I'll probably dive in and do the update next week, and let you know. 

[Przemyslaw Rudzki]

Any progress on the issue? I am trying to use version 3.3.2 and it just does not get started. 

[Joe Zulli]

So this is hilarious. I just upgraded from my special patch version of 3.2.3 to their newest agent version 3.4.0, and for the 3rd time in as many tries, my monitoring has COMPLETELY stopped working. This is the third time in as many tries of updating the agent. Perhaps these guys aren't huge into regression testing. I don't know, but as you can probably tell, I'm super annoyed. Luckily, I still have my special patched version to fall back to. I'm opening up a support ticket. You should too.

Joe

[Ben Castle]

Im trying to trial new relic on a 2.2.1 Play App with no luck at all. 

Have gone through 3.3.X new relic jars up to a 3.4.1.RC1 jar provided by new relic support - and the only Web Transaction being logged is /controllers.Assets.at.

Not one thing fixed with any upgraded agent jar file.

Extremely frustrating as our trial is about to expire so looks like new relic won't be getting our business. If anyone gets a working new relic agent from 3.4.X branch , please yell.

Ben

[Joe Zulli]

Here's the latest update on my end. I just spoke with one of the support engineers @ NewRelic and they told me that the fix they had made for me previously, accidentally didn't get merged in. So I have the only working version in existence! Yay! They're merging it now and it will be released as part of 3.4.1 either today or tomorrow. As soon as it's out, I'll try it out and let you guys know. Assuming you all are experiencing the same problem as me, this should fix it.

We're close :)

Joe

[Przemyslaw Rudzki]

Hi,

I have just downloaded version 3.4.1 and could not make it to work.

I have tried the following two sets of JAVA_OPTS:

heroku config:set JAVA_OPTS="-Xmx384m -Xss512k -XX:+UseCompressedOops -J-javaagent:newrelic/newrelic.jar"

heroku config:set JAVA_OPTS="-Xmx384m -Xss512k -XX:+UseCompressedOops -javaagent:newrelic/newrelic.jar"

This is no go. If anybody got it working please let know what parameters.

BTW. is there any other way to verify that the agent actually got started other than looking on the application console on new relic web site? Anything in logs?

/p

[René Grüner Vangsgaard]

I am running 3.4.1 now with somewhat success. I am using CloudBees, who take care of packaging and enabling the newrelic.jar.

It looks like data is sent to New Relic. The Web transactions are shown, but they give little insight. To me it looks like the data are not parsed/understood in a meaningful way by New Relic. The controller method linked to here (screen shot) should spend most of its time in the database: http://i.imgur.com/hlLQMxm.png

The same goes for another controller method which spends most of its time calling WS (HTTP call) - but the "drill down" is not useful as well.

I have reported this to New Relic.

[Ben Castle]

For our case currently using a customer new relic agent 3.4.1.74572 - standard play 2.2.X java actions we still aren't seeing any web transactions recorded. 

New relic support got back to me with following : 

The agent should be picking up your web transactions in our Netty instrumentation but it appears they are not hitting the "unfoldAndFireMessageReceived" method in "org/jboss/netty/handler/codec/frame/FrameDecoder". Can you give us some insight into how Netty is handling you web requests? A minimal test case that reproduces the problem would be ideal. If not, running a thread profiling session while running load would be helpful

So I think I will put together a very barebones play project to see what is going on. 

--
You received this message because you are subscribed to a topic in the Google Groups "play-framework" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/play-framework/HT0OAG7S1aA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to play-framewor...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[René Vangsgaard]

I am running the Scala version of Play, which apparently hits unfoldAndFireMessageReceived

[Joe Zulli]

For what it's worth, I just upgraded to 3.4.1 and everything is finally working nicely for me. I'm running the Scala version of Play. 

Joe 

[Przemyslaw Rudzki]

Joe,

What configuration have you used for heroku? Are you running on heroku?

/p

[René Vangsgaard]

So you do not get the same useless info as I linked to in my previous post? The Web transaction view actually tells you what it is doing?

[Joe Zulli]

Przemyslaw - I'm not using Heroku, just directly using ec2. The only thing I did to enable newrelic is adding the following to my startup script: -J-javaagent:/usr/local/newrelic/newrelic.jar

René - I'm getting that too. I guess I have a much lower standard for what is useful :). If you ignore the garbage at the top, the stuff below it looks accurate to me (for my app, at least), and that's what I'm using to do my optimizations. 

Joe

[amit sharma]

Hey Guys,

I am also facing the same issue on heroku. Find more details on this link

http://stackoverflow.com/questions/21231687/adding-new-relic-addon-to-heroku-play-framework-2-2-1-java-app

Any pointers will be appreciated..

Thanks

Amit

[Bruno Batarelo]

Hello. I had the same problem and I figured it out few days ago.

Some facts:
1 - when deployed to Heroku, pre 2.2.x applications were storing dependencies in directory target/staged/ (example: target/staged/newrelic-agent-3.1.0.jar)
2 - Play start script that was different, differently named and in different location and previously it was taking into account -javaagent in JAVA_OPTS
3 - almost nobody needed Procfile because the one inferred by Heroku was enough

From Play 2.2 situation on Heroku looks like this:
1 - dependencies are now here: target/universal/stage/lib/ and are named differently (example: target/universal/stage/lib/com.newrelic.agent.java.newrelic-agent-3.4.0.jar)
2 - it is now possible to pass -javaagent and other params but with -J prefix according to the script to the JVM, but for that you would probably need to modify start script. I have no idea if it would be overwritten by default one on next deployment so I didn't even test it. Adding -J to JAVA_OPTS didn't work for me so I question whether that environment variable is even needed anymore. Didn't test that so I can't claim anything.
3 - Procfile is the solution

Basically what you need to do is create a Procfile in the root of your project and make it look something like this (single line):
web: target/universal/stage/bin/YOUR_APP_NAME -Dhttp.port=${PORT} -J-javaagent:target/universal/stage/lib/com.newrelic.agent.java.newrelic-agent-3.4.0.jar -J-Dnewrelic.config.file=conf/newrelic.yml

YOUR_APP_NAME should be replaced by, well, your application name. Actually this whole part "web: target/universal/stage/bin/YOUR_APP_NAME -Dhttp.port=${PORT}" you can copy and paste from Heroku application management page.
Obviously, you have to pay attention to the newrelic jar varsion you are using so when you change your sbt configuration in order to get yourself a newest jar, you'll have to change Procfile as well.

Hope it helps,
Bruno

[Przemyslaw Rudzki]

Hi Bruno,

Thx for the heads up. I have tried your solution and had to do a minor change. Running 

heroku run 'ls target/universal/stage/lib/com.newrelic.*'

showed me only the api jar i.e. target/universal/stage/lib/com.newrelic.agent.java.newrelic-api-3.4.1.jar and I am not sure if this is complete package or just interfaces or sth. I actually added newrelic directory (as suggested in the heroku guide) in the root of my project and commited it to git repo. Then I have modified your Procfile like this: 

web: target/universal/stage/bin/YOUR_APP_NAME -Dhttp.port=${PORT} -J-javaagent:/app/newrelic/newrelic.jar -J-Dnewrelic.config.file=/app/newrelic/newrelic.yml

It seems to be working now. In the above solution I will not have to worry about changing paths of Procfile when upgrading for the next versions of agent. I am going to play a bit with JAVA_OPTS deal later on but for now I am pretty happy it is running.

Once again thx.

/p

--

You received this message because you are subscribed to a topic in the Google Groups "play-framework" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/play-framework/HT0OAG7S1aA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to play-framewor...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--

Przemyslaw Rudzki | CEO @ netbulls.pl | +48 884 980 357

Al. 1000-lecia P.P. 2 / 602-603, 15-111 Białystok, Poland

[jmay...@eco2market.com]

Thks Przemyslaw, you're setup is the one that worked for me !

Jean

[Ömer Faruk Gül]

One of the critical problems with play2.2.1 and Heroku is that Play does not read JAVA_OPTS system variable, instead you have to write everything to Procfile. 

I have a similar question on Stackoverflow: http://stackoverflow.com/questions/22056013/passing-correct-jvm-arguments-to-play-2-2-1-app-on-heroku

But my problem is a little bit different, new relic works perfectly well but I can't pass -Xmx and -Xms variables.

[Will Sargent]

Did using lowercase "java_opts" solve your issue? http://stackoverflow.com/a/22066773

Will Sargent

Consultant, Professional Services

Typesafe, the company behind Play Framework, Akka and Scala

--
You received this message because you are subscribed to the Google Groups "play-framework" group.
To unsubscribe from this group and stop receiving emails from it, send an email to play-framewor...@googlegroups.com.

[Amit Sharma]

It did for me . 

Thanks,

Amit

[Alex Suzuki]

For me, setting the javaagent path to -javagent:/app/newrelic/newrelic.jar solved the issue.

[Sean Sekora]

These steps worked for me http://playframework.wiki/index.php?title=Heroku_Deployment_-_New_Relic_Add-on

[Amit Sharma]

try the accpeted solution here - http://stackoverflow.com/questions/21231687/adding-new-relic-addon-to-heroku-play-framework-2-2-1-java-app

Thanks,

Amit

--

You received this message because you are subscribed to a topic in the Google Groups "play-framework" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/play-framework/HT0OAG7S1aA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to play-framewor...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.