Reading Props vs System.getProperty

[Joe Barnes]

I've run into a difficulty while using the lift-omniauth plugin which reads deployment-specific info from Props.  In my project, it would be better if it read some of those from System.properties.  Does anyone know of a way to have the Props read also from the JVM system properties?  My understanding is they are completely separate, and I need to change the code that reads the properties if I want to prefer Props vs System.getProperties.  I can certainly update that plugin to work this way, but I'd like to first make sure there isn't a clean way to do this.

Thanks,

Joe

[Diego Medina]

Do you use System properties in your deployment because you don't have a way to modify a production.props after you deploy your app?

If that is the case, there *is* a way to specify a path to the props file, that lives outside the jar/war file, so you can keep things in props file, and not worry about System.Properties

Thanks

  Diego

--
--
Lift, the simply functional web framework: http://liftweb.net
Code: http://github.com/lift
Discussion: http://groups.google.com/group/liftweb
Stuck? Help us help you: https://www.assembla.com/wiki/show/liftweb/Posting_example_code
 
---
You received this message because you are subscribed to the Google Groups "Lift" group.
To unsubscribe from this group and stop receiving emails from it, send an email to liftweb+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--

Diego Medina
Lift/Scala consultant
di...@fmpwizard.com
http://fmpwizard.telegr.am

[Joe Barnes]

Hey Diego,

Thanks as always for replying.  To be more specific, the nature of the plugin requires the application's URL to be specified.  Furthermore, I'm deploying the application in AWS where I can easily give a system property, but not so easily give a file.  If I could use the system properties to specify the base URL of the application, it will be a cinch for me to move it to different "environments" (per AWS terminology).  But as you noted, these properties must be baked into the WAR file.  

I'm inclined to update the plugin because of the nature of these particular properties.  I think of a run mode as a class/type of deployment, not a particular deployment.  A URL belongs to a particular deployment, not a class of deployments.  It should be possible to have two test deployments running, for instance.  As it stands, it is only possible to deploy an app with omniauth 6 times, each in its own run mode.

Joe

You received this message because you are subscribed to a topic in the Google Groups "Lift" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/liftweb/5T_4xm1amr8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to liftweb+u...@googlegroups.com.

[Diego Medina]

OK, just don't remove the props option (I use it :) )

but for a reference, you can specify a props file that lives outside your war file using

    Props.whereToLook = () => ("production.digital.props", () => tryo(new FileInputStream(System.getProperty("Props.path")))) :: Nil

and it is not true that you cannot run two test deployment on diff urls. the way lift looks for files is not just on runmode, it is also on username and hostname, so you can have a lot more prop files :)

Thanks

  Diego

[Austen Holmes]

I created a PropertiesResolver that can be plugged in.  One of my resolvers is an S3Resolver.  The properties resolver is determined via a command-line flag (-Dprops.resolver=s3://my-bucket/path-to-my.props).  The first thing I do in boot (before any props are accessed) is change Props.whereToLook.

 If you're running on aws, you don't even have to pass in the aws access/secret keys.  You can use an instance profile that has read-only permissions to access my-bucket and launch the ec2 instance with the correct permissions.  A nice thing about this is you can combine it with runlevels, so I have s3://my-bucket/production/my.props and s3://my-bucket/staging/my.props and have the application build the correct path at runtime.

[Joe Barnes]

Thanks to the pointers everyone.  I had forgotten that username/hostname can also be used for Lift to find files.  Does anyone know how the hostname is derived?  I'm not sure what that will be since I'm in the AWS cloud.

I may also look into Austen's suggestion regarding utilizing S3.

Hopefully one of these will be straight-forward enough I don't feel a need to update that plugin.  Don't worry Diego, I certainly wouldn't violate the current behavior.  I would make it such that it would act on System properties only if Props isn't defined or something along those lines.

Joe

[Antonio Salazar Cardozo]

On Tuesday, December 17, 2013 1:46:37 PM UTC-5, Joe Barnes wrote:

Thanks to the pointers everyone.  I had forgotten that username/hostname can also be used for Lift to find files.  Does anyone know how the hostname is derived?  I'm not sure what that will be since I'm in the AWS cloud.

It uses InetAddress.getLocalHost.getHostName. I suspect it'll be the internal hostname, but not sure.

Thanks,

Antonio

[Austen Holmes]

If you're using a default amazon ami inside of vpc, you'll get a hostname like "ip-10.0.0.123".

Depending on the ami + container you use, you may get an exception when starting up the server (I don't remember the exception, though), because it doesn't list its hosts name in the hosts file.  Frustrating.  For example, see below (this is using the default tomcat ami with elastic beanstalk):

[ec2-user@ip-172-17-2-176 ~]$ hostname

ip-172-17-2-176

[ec2-user@ip-172-17-2-176 ~]$ cat /etc/hosts

127.0.0.1   localhost localhost.localdomain

I had to write a script to append hostname to localhost when the image is created (otherwise, autoscaling instances won't launch correctly).

I could probably configure dns better for my aws vpc, but haven't really had the time to do it.

-Austen

[Joe Barnes]

I finally got back around to working on these changes.  I have opened a pull request.

Joe

[Joe Barnes]

So, I wanted to start a thread about this, and it turns out I did already in Dec 2013. :)  

I still to this day find myself wishing that Lift's Props would first check System properties.  I continually find it useful to define the defaults I want in the props files baked into the application, and tweak a few parameters from System properties.  

I would love to add a feature in Lift you could configure during boot to tell us to look in system properties or whatever first.  Does anyone perceive a problem doing this?

Joe

[Matt Farmer]

What rules would win out if two keys were defined?

—
Matt Farmer | Blog | Twitter

--
--
Lift, the simply functional web framework: http://liftweb.net
Code: http://github.com/lift
Discussion: http://groups.google.com/group/liftweb
Stuck? Help us help you: https://www.assembla.com/wiki/show/liftweb/Posting_example_code

---

You received this message because you are subscribed to the Google Groups "Lift" group.

To unsubscribe from this group and stop receiving emails from it, send an email to liftweb+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Andreas Joseph Krogh]

På tirsdag 25. august 2015 kl. 14:33:39, skrev Matt Farmer <ma...@frmr.me>:

What rules would win out if two keys were defined?

 

In my experience it's common to let System-properties win.

 

--

Andreas Joseph Krogh

CTO / Partner - Visena AS

Mobile: +47 909 56 963

and...@visena.com

www.visena.com

 

[Austen Holmes]

That's my experience as well. It's like in-line styles in css.

--

--
Lift, the simply functional web framework: http://liftweb.net
Code: http://github.com/lift
Discussion: http://groups.google.com/group/liftweb
Stuck? Help us help you: https://www.assembla.com/wiki/show/liftweb/Posting_example_code

---

You received this message because you are subscribed to a topic in the Google Groups "Lift" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/liftweb/5T_4xm1amr8/unsubscribe.
To unsubscribe from this group and all its topics, send an email to liftweb+u...@googlegroups.com.

[Joe Barnes]

Definitely system properties win.  Props are baked into the application, and system properties give you the opportunity to override at run time.  

We could make something to be configurable to work either way, but I'm struggling to imagine a scenario where you would want the baked in props files to win.  Perhaps if you are defining props files outside of the war/jars.

Joe

[Robert Marcano]

On 08/25/2015 09:10 AM, Joe Barnes wrote:
> Definitely system properties win. Props are baked into the application,
> and system properties give you the opportunity to override at run time.
>
> We could make something to be configurable to work either way, but I'm
> struggling to imagine a scenario where you would want the baked in props
> files to win. Perhaps if you are defining props files outside of the
> war/jars.

What about allowing Props to embed system property references, like

property=${system.property.name}

I am not happy allowing some of my production properties to be changed
at deployment time with a system property. This way you can control
which system properties interact with your code

>
> Joe
>
>
> On Tue, Aug 25, 2015 at 8:07 AM, Austen Holmes <austen...@gmail.com
> <mailto:austen...@gmail.com>> wrote:
>
> That's my experience as well. It's like in-line styles in css.
>
>
> On Tue, Aug 25, 2015, 7:42 AM Andreas Joseph Krogh
> <and...@visena.com <mailto:and...@visena.com>> wrote:
>
> På tirsdag 25. august 2015 kl. 14:33:39, skrev Matt Farmer

> <ma...@frmr.me <mailto:ma...@frmr.me>>:

>
> What rules would win out if two keys were defined?
>
> In my experience it's common to let System-properties win.
> --

> *Andreas Joseph Krogh*

> CTO / Partner - Visena AS

> Mobile: +47 909 56 963 <tel:%2B47%20909%2056%20963>
> and...@visena.com <mailto:and...@visena.com>
> www.visena.com <https://www.visena.com>
> <https://www.visena.com>

>
> --
> --
> Lift, the simply functional web framework: http://liftweb.net
> Code: http://github.com/lift
> Discussion: http://groups.google.com/group/liftweb
> Stuck? Help us help you:
> https://www.assembla.com/wiki/show/liftweb/Posting_example_code
>
> ---
> You received this message because you are subscribed to a topic
> in the Google Groups "Lift" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/liftweb/5T_4xm1amr8/unsubscribe.
> To unsubscribe from this group and all its topics, send an email
> to liftweb+u...@googlegroups.com

> <mailto:liftweb+u...@googlegroups.com>.

> For more options, visit https://groups.google.com/d/optout.
>
> --
> --
> Lift, the simply functional web framework: http://liftweb.net
> Code: http://github.com/lift
> Discussion: http://groups.google.com/group/liftweb
> Stuck? Help us help you:
> https://www.assembla.com/wiki/show/liftweb/Posting_example_code
>
> ---
> You received this message because you are subscribed to a topic in
> the Google Groups "Lift" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/liftweb/5T_4xm1amr8/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> liftweb+u...@googlegroups.com

> <mailto:liftweb+u...@googlegroups.com>.

> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> --
> Lift, the simply functional web framework: http://liftweb.net
> Code: http://github.com/lift
> Discussion: http://groups.google.com/group/liftweb
> Stuck? Help us help you:
> https://www.assembla.com/wiki/show/liftweb/Posting_example_code
>
> ---

> You received this message because you are subscribed to the Google
> Groups "Lift" group.

> To unsubscribe from this group and stop receiving emails from it, send
> an email to liftweb+u...@googlegroups.com

> <mailto:liftweb+u...@googlegroups.com>.

[Austen Holmes]

I think the thing to recognize is these are system properties (-D<value>), not environment variables, so they are explicitly passed on the command-line.  For our application, we run multiple environments in production run-mode (for a training site, for example), but we change one or two things (api keys, where emails route to, etc).  We also use AWS elastic beanstalk, which provides a nice facility to pass in system properties.

[Joe Barnes]

That's a cool idea too.  I do a similar trick when deploying Lift to Heroku.  In production props, I have jetty.port=$PORT and if I see a $, I read the environment variable $PORT (not java system property).  This is necessary with Heroku because you only get assigned a port to run your web app at deployment time.

Joe

To unsubscribe from this group and all its topics, send an email to liftweb+u...@googlegroups.com.

[ti com]

On Tue, Aug 25, 2015 at 9:54 AM Robert Marcano <rob...@marcanoonline.com> wrote:

On 08/25/2015 09:10 AM, Joe Barnes wrote:
> Definitely system properties win.  Props are baked into the application,
> and system properties give you the opportunity to override at run time.
>
> We could make something to be configurable to work either way, but I'm
> struggling to imagine a scenario where you would want the baked in props
> files to win.  Perhaps if you are defining props files outside of the
> war/jars.

What about allowing Props to embed system property references, like

property=${system.property.name}

this sounds almost like typesafe config

 

To unsubscribe from this group and stop receiving emails from it, send an email to liftweb+u...@googlegroups.com.

[Arseny Tolmachev]

I use typesafe config in my project as well.

[Joe Barnes]

Alright folks, when you get a moment, take a look below at my first stab at stubbing some new methods on Props.  I feel we can implement the proposed functionality in these three methods.  We have prepend() for placing properties BEFORE looking in the usual *.props files and append() for looking up AFTER *.props (thus giving the Lift developer control over conflict resolution).  Also, we have appendInterpolator() for creating a stack of maps for further looking up properties when Lift finds a ${something} within a value.

This is of course very incomplete.  For instance, I'll have to add some overloaded signatures for System.props, System.env and so forth to make it all work.  I'm posting to see how everyone likes the direction I have in mind while I play with it in my branch.

Here are the signatures in Props:

def prepend(props:Map[String, String]):Unit = {}

def append(props:Map[String, String]):Unit = {}

def appendInterpolator(props:Map[String, String]):Unit = {}

I also defined some specs to demonstrate expected behavior.  

# test.default.props
default.jndi.name=from_props

# Used in net.liftweb.util.PropsSpecs
jetty.port=${PORT}
db.url=jdbc:mysql://${DB_HOST}:${DB_PORT}/MYDB

"Prefer prepended properties to the test.default.props" in {
  Props.prepend(Map("jetty.port" -> "8080"))
  val port = Props.getInt("jetty.port")

  port must_== Full(8080)
}

"Find properties in appended maps when not defined in test.default.props" in {
  Props.append(Map("new.prop" -> "new.value"))
  val prop = Props.get("new.prop")

  prop must_== Full("new.value")
}

"Not interpolate values when no interpolator is given" in {
  val port = Props.get("jetty.port")

  port must_== Full("${PORT}")
}

"Interpolate values from the given interpolator" in {
  Props.appendInterpolator(Map("PORT" -> "8080"))
  val port = Props.getInt("jetty.port")

  port must_== Full(8080)
}

Joe

[Joe Barnes]

Opened PR 1722 for the property lookup enhancement.

Joe

[Joe Barnes]

Opened PR 1722 for the property lookup enhancement.

Joe

On Sunday, 30 August 2015 11:06:47 UTC-5, Joe Barnes wrote: