[2.x] Static assets handling performance

[Guillaume Bort]

Hi all,

We just introduced a new way of detecting file changes in static asset
files. It now uses the jnotify API to detect filesystem changes
instead of checking the last modified date for each file at each
request. I hope that it will fix all issues related to static assets
reloading performance in development mode.

The commit is in the master:

https://github.com/playframework/Play20/commit/e170e85b968e71d5ac0370dd4faec36faa57ad01

Please check and comment if it properly fixes your problems.

Thanks.

--
Guillaume Bort, http://guillaume.bort.fr

[luiza...@gmail.com]

In ubuntu 12.04 64 bits I have this error: libjnotify.so: classe ELF errada: ELFCLASS32 (Possible cause: architecture word width mismatch)

Luiz Fernando Alvino

--
You received this message because you are subscribed to the Google Groups "play-framework" group.
To post to this group, send email to play-fr...@googlegroups.com.
To unsubscribe from this group, send email to play-framewor...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/play-framework?hl=en.

[Guillaume Bort]

Ok, the 64 bits version of the libjnotify.so is missing. I will fix it.

[C. Mundi]

I look forward to using the new capability, because I like the idea of serving static assets without a separate dedicated server.   Even if the new code works great for everyone today, some strategic caution may be a good idea.

Jnotify is developed by someone who has a deep understanding of the platform-specific issues.   I do not envy that task.  See the caveats in the per-platform notes prominently linked on the jnotify homepage. 

I look forward to a day when a standardized cross-platform  solution is widely implemented.  Until then, kits like jnotify will help in many instances, but probably not without platform-specific pains.  Meanwhile I hope very much that Play will continue first-class support for robust solutions, even at the expense of efficiency.

Carlos

[Guillaume Bort]

Can you try again with my latest fix?

Do forget to clean the ivy cache in
$PLAY/repository/cache/net.contentobjects.jnotify/

On Thu, Jul 19, 2012 at 11:07 AM, luiza...@gmail.com
<luiza...@gmail.com> wrote:

[Guillaume Bort]

> I look forward to a day when a standardized cross-platform solution is
> widely implemented.

This feature (WatchEvents) is available in NIO2 in Java 7. But for now
we will stick with JNotify for compatibility reasons.

[C. Mundi]

On Thu, Jul 19, 2012 at 11:47 AM, Guillaume Bort <guillau...@gmail.com> wrote:

> I look forward to a day when a standardized cross-platform  solution is
> widely implemented.

This feature (WatchEvents) is available in NIO2 in Java 7. But for now
we will stick with JNotify for compatibility reasons.

Well shame on me for missing that.  Thank you for pointing that out. 

[luiza...@gmail.com]

It's working with last commit. Thanks!

Luiz Fernando Alvino

--

[Stefan]

hi guillaume!

I'm facing one problem when I want to run a cleaned project.
So if I run "play clean-all" and afterwards "play ~run" or "play run" the process stops with the following error:

[error] net.contentobjects.jnotify.linux.JNotifyException_linux: Error watching MYPROJECTPATH/target/scala-2.9.1/src_managed/main : No such file or directory
[error] Use 'last' for the full log.

I could bypass this by initially compiling and starting the server with the "start" command.
By this time "run" or "~run" work without problems.

thanks
stefan

>> To post to this group, send email to play-framework@googlegroups.com.

>> To unsubscribe from this group, send email to

>> play-framework+unsubscribe@googlegroups.com.

>> For more options, visit this group at
>> http://groups.google.com/group/play-framework?hl=en.
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "play-framework" group.

> To post to this group, send email to play-framework@googlegroups.com.

> To unsubscribe from this group, send email to

> play-framework+unsubscribe@googlegroups.com.

[Guillaume Bort]

Stefan,

Thank you for your report. This problem doesn't appear on MacOS so I
can't reproduce it myself. But I think that my latest commit fixes it.
Can you please check and confirm?

>> >> To post to this group, send email to play-fr...@googlegroups.com.

>> >> To unsubscribe from this group, send email to

>> >> play-framewor...@googlegroups.com.

>> >> For more options, visit this group at
>> >> http://groups.google.com/group/play-framework?hl=en.
>> >>
>> >
>> > --
>> > You received this message because you are subscribed to the Google
>> > Groups
>> > "play-framework" group.

>> > To post to this group, send email to play-fr...@googlegroups.com.

>> > To unsubscribe from this group, send email to

>> > play-framewor...@googlegroups.com.

>> > For more options, visit this group at
>> > http://groups.google.com/group/play-framework?hl=en.
>>
>>
>>
>> --
>> Guillaume Bort, http://guillaume.bort.fr
>

> --
> You received this message because you are subscribed to the Google Groups
> "play-framework" group.

> To view this discussion on the web visit
> https://groups.google.com/d/msg/play-framework/-/UKJ2nBZc8E0J.
>
> To post to this group, send email to play-fr...@googlegroups.com.

> To unsubscribe from this group, send email to

> play-framewor...@googlegroups.com.

[Stefan]

yes that fixed it for me!

awesome work, thank you!

>> >> To post to this group, send email to play-framework@googlegroups.com.

>> >> To unsubscribe from this group, send email to

>> >> play-framework+unsubscribe@googlegroups.com.

>> >> For more options, visit this group at
>> >> http://groups.google.com/group/play-framework?hl=en.
>> >>
>> >
>> > --
>> > You received this message because you are subscribed to the Google
>> > Groups
>> > "play-framework" group.

>> > To post to this group, send email to play-framework@googlegroups.com.

>> > To unsubscribe from this group, send email to

>> > play-framework+unsubscribe@googlegroups.com.

>> > For more options, visit this group at
>> > http://groups.google.com/group/play-framework?hl=en.
>>
>>
>>
>> --
>> Guillaume Bort, http://guillaume.bort.fr
>
> --
> You received this message because you are subscribed to the Google Groups
> "play-framework" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/play-framework/-/UKJ2nBZc8E0J.
>

> To post to this group, send email to play-framework@googlegroups.com.

> To unsubscribe from this group, send email to

> play-framework+unsubscribe@googlegroups.com.

[Guillaume Bort]

Any chance that you can try to debug the code on Windows?

It looks like this code fail:

https://github.com/playframework/Play20/blob/master/framework/src/sbt-plugin/src/main/scala/PlayReloader.scala#L36

However it is very simple, I'm not sure how it can fail.

On Sat, Jul 21, 2012 at 1:33 AM, ucin <uc...@bakoel.es> wrote:
> After update with latest master branch and build my local repo, I got this
> error when running my app
>
> [web2] $ run
>
> [error] Missing JNotify?

> [error] Use 'last' for the full log.

> [web2] $
>
> and using last command
>
> [web2] $ last
> [debug] Running task... Cancelable: true, max worker threads: 4, check
> cycles: false
> [debug] Running task... Cancelable: true, max worker threads: 4, check
> cycles: false
> [debug] Running task... Cancelable: false, max worker threads: 4, check
> cycles: false
> java.lang.RuntimeException: Missing JNotify?
> at scala.sys.package$.error(package.scala:27)
> at sbt.PlayReloader$$anon$2$$anonfun$4.apply(PlayReloader.scala:40)
> at sbt.PlayReloader$$anon$2$$anonfun$4.apply(PlayReloader.scala:40)
> at scala.Option.getOrElse(Option.scala:108)
> at sbt.PlayReloader$$anon$2.jnotify(PlayReloader.scala:40)
> at sbt.PlayReloader$$anon$2$$anonfun$5.apply(PlayReloader.scala:87)
> at sbt.PlayReloader$$anon$2$$anonfun$5.apply(PlayReloader.scala:87)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:194)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:194)
> at
> scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scala:59)
> at scala.collection.immutable.List.foreach(List.scala:45)
> at
> scala.collection.TraversableLike$class.map(TraversableLike.scala:194)
> at scala.collection.immutable.List.map(List.scala:45)
> at sbt.PlayReloader$$anon$2.<init>(PlayReloader.scala:87)
> at sbt.PlayReloader$class.newReloader(PlayReloader.scala:17)
> at sbt.PlayProject$.newReloader(PlayProject.scala:11)
> at
> sbt.PlayCommands$$anonfun$48$$anonfun$50.reloader$1(PlayCommands.scala:497)
> at
> sbt.PlayCommands$$anonfun$48$$anonfun$50.apply(PlayCommands.scala:503)
> at
> sbt.PlayCommands$$anonfun$48$$anonfun$50.apply(PlayCommands.scala:438)
> at scala.Either$RightProjection.map(Either.scala:285)
> at sbt.PlayCommands$$anonfun$48.apply(PlayCommands.scala:438)
> at sbt.PlayCommands$$anonfun$48.apply(PlayCommands.scala:418)
> at
> sbt.Command$$anonfun$sbt$Command$$apply1$1$$anonfun$apply$6.apply(Command.scala:74)
> at sbt.Command$.process(Command.scala:92)
> at
> sbt.MainLoop$$anonfun$next$1$$anonfun$apply$1.apply(Main.scala:121)
> at
> sbt.MainLoop$$anonfun$next$1$$anonfun$apply$1.apply(Main.scala:121)
> at sbt.State$$anon$1.process(State.scala:154)
> at sbt.MainLoop$$anonfun$next$1.apply(Main.scala:121)
> at sbt.MainLoop$$anonfun$next$1.apply(Main.scala:121)
> at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:18)
> at sbt.MainLoop$.next(Main.scala:121)
> at sbt.MainLoop$.run(Main.scala:114)
> at sbt.MainLoop$$anonfun$runWithNewLog$1.apply(Main.scala:103)
> at sbt.MainLoop$$anonfun$runWithNewLog$1.apply(Main.scala:100)
> at sbt.Using.apply(Using.scala:25)
> at sbt.MainLoop$.runWithNewLog(Main.scala:100)
> at sbt.MainLoop$.runAndClearLast(Main.scala:83)
> at sbt.MainLoop$.runLoggedLoop(Main.scala:67)
> at sbt.MainLoop$.runLogged(Main.scala:60)
> at sbt.xMain.run(Main.scala:33)
> at xsbt.boot.Launch$.run(Launch.scala:54)
> at xsbt.boot.Launch$$anonfun$explicit$1.apply(Launch.scala:43)
> at xsbt.boot.Launch$.launch(Launch.scala:68)
> at xsbt.boot.Launch$.apply(Launch.scala:14)
> at xsbt.boot.Boot$.runImpl(Boot.scala:25)
> at xsbt.boot.Boot$.main(Boot.scala:15)
> at xsbt.boot.Boot.main(Boot.scala)
> [error] Missing JNotify?

> [error] Use 'last' for the full log.
>

> I'm sure the jnotify-0.94.jar is in my ivy cache and the build-repository
> are ok.

>
> --
> You received this message because you are subscribed to the Google Groups
> "play-framework" group.
> To view this discussion on the web visit

> https://groups.google.com/d/msg/play-framework/-/flBizvp-OVEJ.
>
> To post to this group, send email to play-fr...@googlegroups.com.

> To unsubscribe from this group, send email to

> play-framewor...@googlegroups.com.

> For more options, visit this group at
> http://groups.google.com/group/play-framework?hl=en.

[ucin]

Sorry Guillaume, but the error is gone now, and I can't reproduce it again.
I'm using the 'clean' command before 'build-repository'.

My app are running well now and all my tests are ok. Thank you.

[Guillaume Bort]

Ok, I will fix this. Thank you.

On Tue, Jul 31, 2012 at 8:01 AM, Mike Bryant <mike...@gmail.com> wrote:
> I just spent some time puzzling over this, after updating Play 2.0 to the
> latest master. For some reason my repository copy of jnotify is just called:
>
>>
>> play-2.1-SNAPSHOT/framework/../repository/local/net.contentobjects.jnotify/jnotify/0.94/jars/jnotify.jar
>
>
> ... so that the line:
>
>> .find(_.contains("jnotify-"))
>
>
> was failing until I took out the trailing hyphen. I do not understand why
> the jar file name has changed.

>
>
> On Sunday, 22 July 2012 20:52:55 UTC+1, Guillaume Bort wrote:
>>
>> Any chance that you can try to debug the code on Windows?
>>
>> It looks like this code fail:
>>
>>
>> https://github.com/playframework/Play20/blob/master/framework/src/sbt-plugin/src/main/scala/PlayReloader.scala#L36
>>
>> However it is very simple, I'm not sure how it can fail.
>>
>
>
>

> --
> You received this message because you are subscribed to the Google Groups
> "play-framework" group.
> To view this discussion on the web visit

> https://groups.google.com/d/msg/play-framework/-/vtcOpQISrp8J.

[Martin]

Hey Gillaume,

I just updated to the latest version of 2.1-Snapshot including your latest commit and now I get the error described above.

Maybe I'm wrong, but your fix can't work. I've added a nooby println statement for debugging whats wrong and find out, that all Files which are in the list generated by

this.getClass.getClassLoader.asInstanceOf[java.net.URLClassLoader].getURLs

are starting with the absolute path ... (I'm using MacOS). I think you were looking for somthing like that (what works for me):

var jnotifyJarFile = this.getClass.getClassLoader.asInstanceOf[java.net.URLClassLoader].getURLs
          .map(_.getFile)
          .find{ filename =>
              filename.substring(filename.lastIndexOf(java.io.File.separator) + 1).startsWith("jnotify")
           }
          .map(new File(_))
          .getOrElse(sys.error("Missing JNotify?"))

Regards,
Martin

Am Dienstag, 31. Juli 2012 18:54:44 UTC+2 schrieb Guillaume Bort:

Ok, I will fix this. Thank you.

On Tue, Jul 31, 2012 at 8:01 AM, Mike Bryant <mike...@gmail.com> wrote:
> I just spent some time puzzling over this, after updating Play 2.0 to the
> latest master. For some reason my repository copy of jnotify is just called:
>
>>
>> play-2.1-SNAPSHOT/framework/../repository/local/net.contentobjects.jnotify/jnotify/0.94/jars/jnotify.jar
>
>
> ... so that the line:
>
>> .find(_.contains("jnotify-"))
>
>
> was failing until I took out the trailing hyphen. I do not understand why
> the jar file name has changed.
>
>
> On Sunday, 22 July 2012 20:52:55 UTC+1, Guillaume Bort wrote:
>>
>> Any chance that you can try to debug the code on Windows?
>>
>> It looks like this code fail:
>>
>>
>> https://github.com/playframework/Play20/blob/master/framework/src/sbt-plugin/src/main/scala/PlayReloader.scala#L36
>>
>> However it is very simple, I'm not sure how it can fail.
>>
>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "play-framework" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/play-framework/-/vtcOpQISrp8J.
>

> To post to this group, send email to play-framework@googlegroups.com.

> To unsubscribe from this group, send email to

> play-framework+unsubscribe@googlegroups.com.

[Ben McCann]

I also am having difficulty with JNotify now

[connectifier] $ run

[error] Missing JNotify?

[error] Use 'last' for the full log.

[connectifier] $ last

[Guillaume Bort]

Yes my fault, I will correct this now. 

To view this discussion on the web visit https://groups.google.com/d/msg/play-framework/-/gcTRwhbPufMJ.
To post to this group, send email to play-fr...@googlegroups.com.
To unsubscribe from this group, send email to play-framewor...@googlegroups.com.

[Ben McCann]

It works now.  Thanks!

[Martin]

Hey Gillaume,

I don't know implementation details of URLClassLoader, but maybe your fix won't work for Windows. An absolute Path in Windows wouldn't contain "/jnotify" because the seperator in Windows is a backslash. Thats why I used java.io.File.separator.

So  "_.contains( java.io.File.separator + "jnotify")" would be better. Or _.matches(regex) ... but mybe I'm wrong.

Regards,
Martin

[Tom Bocklisch]

I am also having problems with the libjnotify.so. When I try to compile our project, I get the following error:

[error] java.lang.UnsatisfiedLinkError: xyz/target/native_libraries/64bits/libjnotify.so: /lib/libc.so.6: version `GLIBC_2.12' not found (required by xyz/target/native_libraries/64bits/libjnotify.so)

Any solution for that?

Regards,

Tom

[Ryan Means]

I can also verify this. Same error as Tom when pulling from master on 9/5/2012.

[error] java.lang.UnsatisfiedLinkError: /opt/play-apps/directory/target/native_libraries/32bits/libjnotify.so: /lib/tls/i686/cmov/libc.so.6: version `GLIBC_2.12' not found (required by /opt/play-apps/directory/target/native_libraries/32bits/libjnotify.so)

[error] Use 'last' for the full log.

Seeing that this is working for others, I wonder if this is due to the age of my linux distro. We have been developing on Ubuntu 8.04 LTE VM's on our desktop. We are upgrading soon to 10.04 (12.04 is too "new" for our Operations dept to feel comfortable with it in a production environment), so hopefully that will resolve the issue. Even though 8.04 is old, it's still supported by Ubuntu until April next year...

[Ryan Means]

I was able to resolve this by upgrading my OS to Ubuntu 12.04. This whole problem seems kind of backwards to me. Going with JNotify instead of using Java's 7 built in method to do this so you could have backwards compatibility for Java 6 users.. but in reality, you have to upgrade to a new OS to get JNotify to even work. If you upgrade to a new OS, the chances are you'll also upgrade to Java 7 at the same time (like we did).

For this issue to really be backwards compatible, it would seem to me that there needs to be some way to configure Play to use JNotify or not to use JNotify so that users of older OS's can still use Play with the understanding that they will have downgraded performance.

All said - I realize this is the master, and if I really wanted to do something about it, I should submit a pull request. I'm happy with 12.04 for now.

Thanks!

[Guillaume Bort]

The problem is that most implementation of JDK7 doesn't implement
properly this feature. At least on MacOS it is so slow that it is
completely unusable. So for new JNotify is still required.

But I agree, we should support working without JNotify on platform
that doesn't support it (I mean without file change detection, so
scanning the whole project for each request).

> --
> You received this message because you are subscribed to the Google Groups
> "play-framework" group.
> To view this discussion on the web visit

> https://groups.google.com/d/msg/play-framework/-/voTnCudFOeIJ.
>
> To post to this group, send email to play-fr...@googlegroups.com.

> To unsubscribe from this group, send email to

> play-framewor...@googlegroups.com.

> For more options, visit this group at
> http://groups.google.com/group/play-framework?hl=en.

[Daniel Berndt]

Hi,

I just ran into the same issue ("version `GLIBC_2.12' not found") on our staging server running Debian 6.0 with a play 2.1 snapshot from early august. (interestingly it ran fine up until today...).

Has this problem been addressed already?

Cheers,

Daniel

[Guillaume Bort]

I have just created a pull request to fix this:

https://github.com/playframework/Play20/pull/511

Please review and comment on the dev group.

To view this discussion on the web visit https://groups.google.com/d/msg/play-framework/-/8gU-58sgzzkJ.

To post to this group, send email to play-fr...@googlegroups.com.
To unsubscribe from this group, send email to play-framewor...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/play-framework?hl=en.

[Daniel Berndt]

Excellent! I just cherry-picked your commit and got it working.

when calling play run, It now outputs:

Cannot load the JNotify native library (/var/lib/jenkins/jobs/MyProject/workspace/target/native_libraries/64bits/libjnotify.so: /lib/libc.so.6: version `GLIBC_2.12' not found (required by /var/lib/jenkins/jobs/MyProject/workspace/target/native_libraries/64bits/libjnotify.so))

Play will check file changes for each request, so expect degraded reloading performace.

--- (Running the application from SBT, auto-reloading is enabled) ---

[info] play - Listening for HTTP on /0:0:0:0:0:0:0:0:9000

(Server started, use Ctrl+D to stop and go back to the console...)

so yes, the work around seems to work.

Thanks a lot, Guillaume!