[2.1-scala] Serving static files that are not resources

[Michel Schinz]

Hi everyone,

I'm writing a Play application (in Scala) that must authenticate users and then, depending on their role and identity, serves them static HTML files (no templating at all, no access to a database, etc). Play's Assets controller almost does what I want, except that it only serves files from the application's resources, not from an arbitrary location in the file system, as I would like.

I don't want to put the HTML files into the application, because I want to be able to change them without re-deploying the application. Additionally I'd like to allow gzipped HTML files, like Asset does, and have proper support for caching.

Right now, it seems that my only option is to duplicate the 200+ lines of Assets.scala into my project, and change 1-2 lines, which isn't ideal. Wouldn't it be a good idea to factor out the code that serves local files, and build Assets on top of that? Or am I missing something?

Thanks for your help,

Michel.

[El Softwarerero (the player formerly known as sun)]

You could also simply use a symbolic link from Play's public directory to your files. Here is how it would work for UNIX like systems: http://linux.die.net/man/1/ln

[Michel Schinz]

2013/7/12 El Softwarerero (the player formerly known as sun) <goo...@suncom.de>

You could also simply use a symbolic link from Play's public directory to your files.

Thanks. I don't know whether it would work (didn't try), but that really looks hackish...

Anyway, I guess I can rephrase my question slightly: Play already offers the sendFile function to send local files. However, sendFile extremely simple and doesn't cache-related headers like If-Modified-Since. Also, it doesn't offer the option of sending a gzipped version of the file automatically if there is one.

Therefore, wouldn't it make sense to move most of the code from Assets.scala to the sendFile function, and then make the assets controller use sendFile directly (like the one for external assets does)? If the answer is yes, I could try to prepare a pull request for that, but being a newcomer to Play (and Web apps, in fact), I just want to make sure that I'm not missing something obvious...

Can anyone tell me whether it sounds like a good idea or not?

Thanks,

Michel.

[Byron Weber Becker]

I have a similar use case.  Due to time pressures, I was going to go with the solution at http://stackoverflow.com/questions/8305853/how-to-render-a-binary-with-play-2-0  But it, too, seems hackish, doesn't deal with the headers and caching, etc.

I'm in the same position as you -- new to Play and web apps -- but your solution sounds good to me.

Byron

[El Softwarerero (the player formerly known as sun)]

Maybe using a symbolic link is hackish but it works very well in my project.

For the purpose of zipping, caching and load balancing most developers in production use a front end http server, in my case Lighttpd. The asset controller is not very sophisticated because it is used mainly for development.

[Michel Schinz]

2013/7/13 El Softwarerero (the player formerly known as sun) <goo...@suncom.de>

Maybe using a symbolic link is hackish but it works very well in my project.

I don't doubt it, and didn't mean to sound dismissive. I'm just surprised that one has to resort to such techniques for a use-case that, to me, seems pretty common.

 

For the purpose of zipping, caching and load balancing most developers in production use a front end http server, in my case Lighttpd. The asset controller is not very sophisticated because it is used mainly for development.

Using a front-end server would help, but only for the part of my application that is public. As I explained in my original message, my application must also serve a pretty large number of protected static pages, which depend on the role and identity of the current user. Therefore, they cannot be served by the front-end server.

Also, I must say that I'd like to avoid as much as possible having a front-end server, as this is one additional piece of software I'd have to install, configure and maintain.

To summarize, what I'd really like to have in Play, I guess, is a smarter sendFile function, one that can handle caching and compression automatically. It looks like I'll have to write my own, unfortunately.

Thanks to you and Byron for the help,

Michel.

[James Ward]

Can you use the ExternalAssets controller?
http://www.playframework.com/documentation/api/2.1.x/scala/index.html#controllers.ExternalAssets$

-James

On 07/12/2013 08:52 AM, Michel Schinz wrote:
> Hi everyone,
>
> I'm writing a Play application (in Scala) that must authenticate users
> and then, depending on their role and identity, serves them static HTML
> files (no templating at all, no access to a database, etc). Play's

> Assets controller /almost/ does what I want, except that it only serves

> files from the application's resources, not from an arbitrary location
> in the file system, as I would like.
>
> I don't want to put the HTML files into the application, because I want
> to be able to change them without re-deploying the application.
> Additionally I'd like to allow gzipped HTML files, like Asset does, and
> have proper support for caching.
>
> Right now, it seems that my only option is to duplicate the 200+ lines
> of Assets.scala

> <https://github.com/playframework/Play20/blob/master/framework/src/play/src/main/scala/play/api/controllers/Assets.scala>

> into my project, and change 1-2 lines, which isn't ideal. Wouldn't it be
> a good idea to factor out the code that serves local files, and build
> Assets on top of that? Or am I missing something?
>
> Thanks for your help,
> Michel.
>

> --
> You received this message because you are subscribed to the Google
> Groups "play-framework" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to play-framewor...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[Jeff]

This should not be used in production. From the scaladoc: "Note that this controller is not intented to be used in production mode and can lead to security issues. Therefore it is automatically disabled in production mode."

[Michel Schinz]

2013/7/15 Jeff <a617...@nepwk.com>

This should not be used in production. From the scaladoc: "Note that this controller is not intented to be used in production mode and can lead to security issues. Therefore it is automatically disabled in production mode."

Indeed. Moreover, ExternalAssets in its current incarnation simply calls sendFile, which has the limitations I mentioned earlier (i.e. no support for gzipped versions of the file to send, no Last-Modified headers, etc).

Michel.

[James Roper]

If it was that simple we would have done it already.  The assets controller caches modification times and etags, which it can do because the assets come from the classpath, which doesn't change.  As soon as the assets are coming from the filesystem, then it can no longer cache things like that.  Additionally, what if the assets controller reads the file length, sets the header, and then the file changes, and then it sends the file?  If the file is shorter, the client will wait forever for the server to send the rest of the file that doesn't exist, if it's longer, the rest of the file will end up being the response to the next request the client makes - which will yield completely unpredictable behaviour.  Maybe you don't care about these risks, but if we provide this support, someone will, and it will be bug, and we'll need to fix it, and fixing it is really not an easy thing to do.

Then there's all the dangers of serving things from the filesystem, including dot dot attacks, special file names such as "com" and "lpd" having special meanings in windows, different escaping rules and path separators for different filesystems, various nuances around symlinks, case sensitivity issues on different platforms, the way different platforms handle modification/deletion of files that are being read concurrently - "factor out the code that serves local files" just doesn't cut it.  The classpath is a very well specified structure that works the same way very predictably across all platforms, that's why we can have a simple (and look at the code, it's not even that simple) controller for serving assets.

As it happens, serving files from the filesystem is not a particularly good way to do things anyway.  For most serious apps on the internet, a CDN is the best way to serve assets.  For those apps that are too small to need a CDN, or that are just for internal use, then we provide a simple mechanism for serving from the classpath.  If you need something else, feel free to implement it, but be aware of the dangers that I've listed above.

Thanks for your help,

Michel.

--

You received this message because you are subscribed to the Google Groups "play-framework" group.
To unsubscribe from this group and stop receiving emails from it, send an email to play-framewor...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--

James Roper
Software Engineer

Typesafe – Build reactive apps!
Twitter: @jroper

[Mike]

I went here redirected from http://stackoverflow.com/questions/18196600/play-framework-assets-not-accessible-in-production-mode by M. Ward.

I need to serve uploaded images. Very strange requirement indeed.

From what I read, Play Framework don't serve such files because:

 - the file size could change and thus the header would be wrong... This is a well known fact that web servers NEVER serve dynamically generated files because they don't know the file size in advance...??? Of course in case it would really matter it is such a difficult task to manage a cache, that Play can't do that...

 - there could be security reason due to rogue file names: yes, it is such a difficult task to normalize/rename uploaded files to avoid any such risk... Play users will be so stupid that they can't do that ??? Or maybe it is about serving uncontrolled paths ? Here I understand easily that checking for such issues is out of reach of Play designers and programmers. But then also this part of the discussion is irrelevant to my problem.

Now if I simply restart the server, I will now get the newly uploaded files, so where is the security problem adressed by refusing to look for new files in an asset folder ???

Solution is either to setup ANOTHER web server for the pics, or store them into a database...
Oh yes in these cases the supposed "size" issue will be solved ????

Come on.

Yet another time completely disgusted by the Play Framework and its developpers.

[Marius Soutier]

Not sure if I should feed the trolls, but uploading an image to your assets folder (if I understand correctly, you don’t give enough details for a complete answer) seems wrong in many ways.

For example see this answer:

http://stackoverflow.com/a/14172965/772743

Many more points come to mind, e.g. how do you scale it? How do you merge new images with the images in your repository?

On 04.04.2014, at 12:28, Mike <miche...@gmail.com> wrote:

I went here redirected from http://stackoverflow.com/questions/18196600/play-framework-assets-not-accessible-in-production-mode by M. Ward.

I need to serve uploaded images. Very strange requirement indeed.

From what I read, Play Framework don't serve such files because:

 - the file size could change and thus the header would be wrong... This is a well known fact that web servers NEVER serve dynamically generated files because they don't know the file size in advance...??? Of course in case it would really matter it is such a difficult task to manage a cache, that Play can't do that...

 - there could be security reason due to rogue file names: yes, it is such a difficult task to normalize/rename uploaded files to avoid any such risk... Play users will be so stupid that they can't do that ??? Or maybe it is about serving uncontrolled paths ? Here I understand easily that checking for such issues is out of reach of Play designers and programmers. But then also this part of the discussion is irrelevant to my problem.

Now if I simply restart the server, I will now get the newly uploaded files, so where is the security problem adressed by refusing to look for new files in an asset folder ???

--
You received this message because you are subscribed to the Google Groups "play-framework" group.
To unsubscribe from this group and stop receiving emails from it, send an email to play-framewor...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[James Ward]

If you really want to you can serve file system based files from Play.
This is really not recommended for all the reasons James Roper outlines.
But there are times when it might make sense.

Here is a simple Scala controller example that is probably full of
security holes and other issues.

val inputStream = // get an InputStream from somewhere
val enumerator: Enumerator[Array[Byte]] = Enumerator.fromStream(inputStream)
Ok.feed(enumerator)

This doesn't have any etag, content type, or cache control header so all
of that would need to be added manually.

Hope that helps.

> --
> You received this message because you are subscribed to the Google
> Groups "play-framework" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to play-framewor...@googlegroups.com

> <mailto:play-framewor...@googlegroups.com>.

[Wei Liu]

Hi Mike,

I want to know how did do with this issue? You just give up Play Framework because of its discustingness or you created your own HTTP cache module? I'm suffering here too. :(

Thank you

[Igmar Palsenberg]

What prevents you from implementing it yourself ? The asset dir should be considered immutable, and with good reasons. If you want to use dynamic assets  / templates, you have to implement it yourself.

I really don't see the problem here.

Igmar