reference.conf/evolutions should be inside a jar / not conf
150 views
Skip to first unread message
Christian Schmitt
May 20, 2016, 3:00:19 PM5/20/16
to Play framework dev
Hello, I stumbled about something while trying to create some installers that don't use the bat/sh files to start play.
A change https://github.com/playframework/playframework/pull/4522/files made to play made it possible to create the jar file for the application, but removing everything from the conf/ folder.
However since Play 2.4/2.5 there is also a reference.conf it's really great to put modules in there since they will mostly not be changed by the application.conf. However when deploying such applications play will generate the jar and remove everything from conf and add it directly in the zip / tar gz in the conf folder I think thats not a good idea for the reference.conf. I think this file should be included directly even in the external-sans file so that the conf folder is not a needed path for the classpath.
The same thing goes for the evolutions and routes file.
Actually the routes file will be inside the conf/ folder but I doubt that it is no needed, so there should be away to exclude it after compilaton.
Actually I also think that there should be a way to include a logback.xml/logconf.xml directly into the external-sans (maybe via some sort of sbt true/false, specifing). But keep everything else away from the external-sans
WDY?
James Roper
May 21, 2016, 9:19:34 AM5/21/16
to Christian Schmitt, Play framework dev
On 21 May 2016 at 05:00, Christian Schmitt <c.sc...@briefdomain.de> wrote:
Hello, I stumbled about something while trying to create some installers that don't use the bat/sh files to start play.A change https://github.com/playframework/playframework/pull/4522/files made to play made it possible to create the jar file for the application, but removing everything from the conf/ folder.However since Play 2.4/2.5 there is also a reference.conf it's really great to put modules in there since they will mostly not be changed by the application.conf.
It doesn't make sense for a Play application to include a reference.conf, libraries include a reference.conf so that they can express defaults and document their configuration format. If you have a Play library, then you create a reference.conf for it, and publish it as a jar file for Play applications to depend on. But if it's a Play app, then everything may as well go in application.conf.
However when deploying such applications play will generate the jar and remove everything from conf and add it directly in the zip / tar gz in the conf folder I think thats not a good idea for the reference.conf.
The reference.conf, and in fact all files from conf, will stay in the main jar file, this change doesn't impact that - if you run sbt package, you'll see sbt creates the regular jar file for the project with all the classpath resources in it. If you wanted your Play app to be used as a library, this is the jar that you would publish, and in fact it is the jar that gets published when you run sbt publish.
What the change is, is when PlayKeys.externalizeResources is true, this jar file is not created when you run sbt dist, instead a different one is created that doesn't have the resources. But that only impacts the dist task, and the dist task is only useful for creating a runnable instance of your application, nothing from it should ever be used by another project, so what does it really matter what is and isn't in the jar file? When you package up an app for distribution, what matters is what ends up on the classpath. In this case, Play offers two options, one is to put all the resources into the conf folder so that it can be edited after deployment, and have the conf folder on the classpath, the other is to leave everything in the jar file. But the distribution is completely irrelevant if you're creating a module, the only thing relevant when creating a module is what's in the main jar file that's created when you run sbt package or sbt publish, and that remains completely unchanged either way.
I think this file should be included directly even in the external-sans file so that the conf folder is not a needed path for the classpath.The same thing goes for the evolutions and routes file.Actually the routes file will be inside the conf/ folder but I doubt that it is no needed, so there should be away to exclude it after compilaton.Actually I also think that there should be a way to include a logback.xml/logconf.xml directly into the external-sans (maybe via some sort of sbt true/false, specifing). But keep everything else away from the external-sansWDY?
The resources that Play will externalize are gathered using the PlayKeys.playExternalizedResources task. So if you want to exclude something from being externalized, you just have to modify that in build.sbt, for example:
PlayKeys.playExternalizedResources ~= (_.filter {
case (_, "reference.conf") => false
case (_, "logback.xml") => false
case _ => true
})
--
You received this message because you are subscribed to the Google Groups "Play framework dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to play-framework-...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Christian Schmitt
May 22, 2016, 6:19:28 AM5/22/16
to Play framework dev, c.sc...@briefdomain.de
Actually thanks for your explantation, however with some things we differ.
The reference.conf could be used for Module bindings and an Application Entry point.
Why I will prefer the reference.conf there has some reasons.
Why I will prefer the reference.conf there has some reasons.
Actually consider your Ops Team should only see the application.conf were they can tweak parameters, but they should never see something like: play.modules.enabled or play.application.loader. The same thing happens when you sell your application. Putting everything in a config file that will be exposed to people that have no knowledge about Guice Modules / Entry Points is a really bad idea and since play has no other way of configuring those I would consider it bad to put that into application.conf.
Another thing is actually the evolutions, I think they directly belong to the application and should not be editable (I know a jar is editable, too but most people wouldn't touch them) and they should directly be put into the jar when running sbt dist
To unsubscribe from this group and stop receiving emails from it, send an email to play-framework-dev+unsub...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
James Roper
May 22, 2016, 10:29:02 PM5/22/16
to Christian Schmitt, Play framework dev
I think that's a misuse of reference.conf that will cause you problems very quickly. While play.modules.enabled can be modified by end users from reference.conf, it's a special case because you're not overriding the value, you're appending to the existing value, which means it doesn't matter what order the configuration files are loaded in or where they come from. Any other configuration options that Play provides, for example, play.application.loader, which you also probably don't want ops people to be concerned with, should not be configured in reference.conf, since Play's own reference.conf already configures it, and there's no defined ordering as to whether yours or Plays is loaded last. If you're doing this now and it works for you, consider that in future something tiny in Play or your app might change, and suddenly Play's default application loader will be used, and it won't be a bug because we've always said that there are no guarantees with classpath ordering in Play.
If you want to separate configuration that ops can change vs configuration that only developers are concerned with, then I'd simply use different configuration files that include each other, eg have an ops.conf that gets included from application.conf. And then if you want to control which files get bundled in the jar and which are configurable post deployment by ops, then use PlayKeys.playExternalizedResources to do that, for example if you just want ops.conf to be externalized, that can be done like so:
PlayKeys.playExternalizedResources := Seq(
baseDirectory.value / "conf" / "ops.conf") -> "ops.conf"
)
I don't think there's any apparent way for the framework to decide what users do and don't want externalised, what seems logical to you may be completely the opposite of what someone else wants, for example you don't want evolutions, but I can imagine some very good use cases for having evolutions separate, since if they fail, you may want to change them at runtime to fix the deployment. This is why when I implemented this I made playExternalizedResources its own task so that people can completely customise what does and what doesn't get bundled into the jar.
To unsubscribe from this group and stop receiving emails from it, send an email to play-framework-...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "Play framework dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to play-framework-...@googlegroups.com.
Ben McCann
May 23, 2016, 1:09:15 AM5/23/16
to James Roper, play-fram...@googlegroups.com, Christian Schmitt
We have a dev.conf and prod.conf that both include a common.conf. We put application logic like modules.enabled in common.conf. And we put the ops stuff in dev.conf and prod.conf. We load prod.conf using a command line option that play supports (I think something like -Dconfig.file=prod.conf). Our prod.conf file is checked into a seperate repo that ops manages.
Greg Methvin
May 23, 2016, 2:29:17 AM5/23/16
to Ben McCann, James Roper, Play framework dev, Christian Schmitt
Yeah, just having separate configuration files, one where ops can change one and devs are responsible for (that includes the other one), is more or less what I've used in my experience (mostly startups). Maybe in your case you want stronger controls, but that could probably be managed by some other means.
Christian Schmitt
May 23, 2016, 2:44:31 AM5/23/16
to Play framework dev, c.sc...@briefdomain.de
Actually regarding the classpath issue we ran into this a few times. But only on windows since there the classpath is too big so we needed to adjust it to have something like -cp "lib\*".
Actually what will happen if my library AND my application has a application.conf and I load a production.conf via -Dconfig.file will it still use all values and if any value will be overriden in production conf, it will be used instead the value of the application.conf?
Hm maybe other's aren't as cautionous as we are, however evolutions should probably not changed directly. We test everything on staging so that any error inside the evolution will be fixed, manual editing those or even outside of a redeployment could be a really really messy thing. But actually maybe you are right on that, not everybody does everything right.
Thanks for your idea's.
Maybe we should work out some idea's and change at least one of those:
to describe one "workflow" regarding that.
To unsubscribe from this group and stop receiving emails from it, send an email to play-framework-dev+unsub...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "Play framework dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to play-framework-dev+unsub...@googlegroups.com.
Christian Schmitt
May 23, 2016, 4:42:35 AM5/23/16
to Play framework dev, c.sc...@briefdomain.de
Actually I played a little bit with it and what I find a little bit akward is that I can't have a application.conf that will be picked up even if I specify a -Dconfig.file.
Currently if I have a play app with libraries it will pick up their application.conf, regarding what I specify with config.file.
So my structure:
app/
conf/application.conf
modules/lib1/app
modules/lib1/conf/application.conf
modules/lib2/app
modules/lib2/conf/application.conf
However the "main" application.conf won't be picked up, even If I rename both application.conf and include them inside the "main" application.conf, when I specify something -Dconfig.file=conf/production.conf.
However the "main" application.conf won't be picked up, even If I rename both application.conf and include them inside the "main" application.conf, when I specify something -Dconfig.file=conf/production.conf.
So this keeps me with 2 possibilities. Either the "production.conf" or any other conf inclueds the application.conf via 'include "application.conf"' or I keep the reference.conf and make a classpath like -cp "lib\myapp-1.0.0;lib\*"
I consider the second one as a no go since a classpath should not specify a order. And the first one actually means that I somehow give a hint about my configuration. There is actually no way of defining a configuration that will be picked up where I can define my modules / default configuration of play that should not be changed. (i.e: play.http.parser.maxMemoryBuffer, play.http.parser.maxDiskBuffer or something ike play.i18n.langs) and where the user won't need any inclue or something other and the file is just inside the classpath.
Jean Helou
May 23, 2016, 4:57:04 AM5/23/16
to Christian Schmitt, Play framework dev
We put
include "application.conf"
as the first line of conf/production.conf ... ;)
cheers
jean
To unsubscribe from this group and stop receiving emails from it, send an email to play-framework-...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "Play framework dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to play-framework-...@googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "Play framework dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to play-framework-...@googlegroups.com.
James Roper
May 23, 2016, 8:12:37 AM5/23/16
to Christian Schmitt, Play framework dev
I would recommend being explicit. If you create a class in the root package in lib1 called Filter, and another class in the root package in lib2 called Filter, do you expect the JVM to load both classes and merge their logic? Of course not. You give them different names, and you explicitly name them. The same goes for configuration files. The fact that the JVM lets you have multiple files with the same name on the classpath is possibly a misfeature, multiple files on the filesystem with the same name don't work, they have to have different names or be in different directories. Allowing it on the classpath introduces hidden magic in the libraries that use it, magic that isn't apparent to you as the end user, and magic that breaks down very quickly, for example as you experienced when you deployed to Windows.
So, keep things simple, use different names for different files, and tell the application exactly how to assemble them together with includes. Don't rely on classpath ordering, because when it breaks (eg because someone changes the name of the artifact so your -cp argument is wrong) and you're no longer there, how is the next guy going to know to look at that classpath argument? Is there any obvious link that the problem they are seeing is linked to that? No. So keep things simple, use different names for different files. This will save not only you, but also the next developer who comes along to maintain your system and hasn't had all the experience you've had in working out how things work, a lot of pain.
To unsubscribe from this group and stop receiving emails from it, send an email to play-framework-...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "Play framework dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to play-framework-...@googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "Play framework dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to play-framework-...@googlegroups.com.
Christian Schmitt
May 24, 2016, 5:26:32 AM5/24/16
to Play framework dev, c.sc...@briefdomain.de
Yeah thanks I'm explicit now.
I also found out that I don't need to use play.modules.enabled. Since Play will load a default Module.class where I just could add other Modules with install() so I don't run into any issue that somebody will disable that.
There is only one thing that could be specified without a Configuration. and that is a ApplicationLoader (I created an issue here: https://github.com/playframework/playframework/issues/6184) with that it would be possible to override Modules without touching the configuration.
The only thing that needs to be inside the config is then only the akka config/http defaults.
To unsubscribe from this group and stop receiving emails from it, send an email to play-framework-dev+unsub...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "Play framework dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to play-framework-dev+unsub...@googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "Play framework dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to play-framework-dev+unsub...@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages