Set configuration values on application.conf from file system (Docker Swarm secrets)

[Mario S]

I have this use case:  I'm deploying Play applications using Docker, and as you know, it's really easy to pass environment variables and set values on application.conf:

application.key.secret = ${?APPLICATION_KEY_SECRET}

But, Docker Swarm works different. It does not load secrets into the containers as environment variables. Instead, it loads them as files like:

/run/secrets/com.company.application.key.secret

Right now, I'm doing this as workaround:

1) Register the secrets on Docker Swarm like

com.company.application.key.secret=SECRET_VALUE

com.company.application.db.password=SECRET_PASSWORD

[...]

   Every secret will be loaded as files into the containers:

/run/secrets/com.company.application.key.secret

/run/secrets/com.company.application.db.password

[...]

 

2) Use an entrypoint.sh file for the Play Application's Docker container. It reads the files and sets environment variables:

export APPLICATION_KEY_SECRET=$(cat /run/secrets/com.company.application.key.secret)

export APPLICATION_DB_PASSWORD=$(cat /run/secrets/com.company.application.db.password)

     [...] 

3) Set the variables on application.conf as usual

application {

key.secret = ${?APPLICATION_KEY_SECRET}

db.password = ${?APPLICATION_DB_PASSWORD}

     [...] 

} 

It works, but as you can see, it's not the best way to do it. So far I understand, the idea of use the file system instead of environment variables is because the last ones are insecure as long as they can be read for any process. In that sense, well, let's say that the workaround is not making things better. 

Proposal: enable a way to set values on application.conf, using the file system. It would be something like this:

application{ 

key.secret = @{?/run/secrets/com.company.application.key.secret}

db.password = @{?/run/secrets/com.company.application.db.password)

[...]

} 

Looking forward for your comments or questions.

Best,

Mario

[Dominik Dorn]

you could try to use the include statement from typesafe/config

https://github.com/typesafehub/config

try something like
application {
 key.secret = include file("/run/secrets/com.company.application.key.secret")
}

--
You received this message because you are subscribed to the Google Groups "Play framework dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to play-framework-dev+unsub...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Dominik Dorn
http://dominikdorn.com | http://twitter.com/domdorn
XING: https://www.xing.com/profile/Dominik_Dorn
LINKEDIN: http://at.linkedin.com/pub/dominik-dorn/66/b42/bb1/

[Mario Sotil]

I tried that configuration but it didn't work... it was a month ago and I don't remember if I did it as you recommend. I will try again.

Thanks,

___________________________
Mario Sotil
NoSQL, Java developer - email: mario...@gmail.com

To unsubscribe from this group and stop receiving emails from it, send an email to play-framework-dev+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Dominik Dorn]

if all fails, you can try do create your own configuration provider like

play.api.inject.ConfigProvider

or 
play.api.inject.ConfigurationProvider

which enhances the configuration with the values read from the file system. 

[Mario Sotil]

Awesome. I didn't know that option. Let me try that too, before close this feature request.

___________________________
Mario Sotil
NoSQL, Java developer - email: mario...@gmail.com

[Dominik Dorn]

It might not be soo trivial, because the configuration is loaded in the BuiltinModule.. you probably have to create your own play.ApplicationLoader that extends play.inject.guice.GuiceApplicationLoader where you then override the bindings for the configuration. 

In the application.conf you then need to set your ApplicationLoader, e.g. like

play.application.loader = "com.actimust.play.spring.SpringApplicationLoader"

( from https://github.com/remithieblin/play-spring-loader , also take a look at the impl they did there)

I'm not 100% sure about the lifecycle, but what I think will happen is that play loads the configuration, then finds the applicationLoader, and in your applicationLoader you can then make sure its using your ConfigurationProvider which has support for reading the files and that one will be used for the rest of the application lifecycle. 

Its actually quite a lot of effort to get "reading a secret from a file instead of environment variable" working.. I think we should create a Ticket for that to fix it in the future. 

[Dominik Dorn]

Also take a look at this:
https://www.playframework.com/documentation/2.5.x/ScalaDependencyInjection#Advanced:-Extending-the-GuiceApplicationLoader

[Mario Sotil]

Yeah, I'm agree with you that develop all of that could be too much to set configuration variables from the file system. 

I think I will keep using my workaround until Play be able to do it. 

I would like to try to develop the improvement on Play. I hope to have some time the next week to take a look.

Thanks for your help and time.

[Dominik Dorn]

The right place to make the adjustment would be

https://github.com/typesafehub/config

as play is just using that as library. 

cheers,

Dominik

[Mario Sotil]

Awesome. I will go there.

Best,

Mario

___________________________
Mario Sotil
NoSQL, Java developer - email: mario...@gmail.com

[Ashley Aitken]

Any progress / update on this?  I couldn't see an issue in typesafe/config on Github?

I agree it would be great to include this functionality in typesafe/config.

Maybe with syntax something like =$(< /path/to/secret/in/file)

So we could do something like:

application.key.secret = "default-value"

application.key.secret = ${?APPLICATION_KEY_SECRET}

application.key.secret = ${< /path/to/secret/in/file}

or perhaps

application.key.secret = "default-value"

application.key.secret = ${?APPLICATION_KEY_SECRET} 

application.key.secret = ${< APPLICATION_KEY_SECRET_FILE}

using the value in the environment variable as the path to the secret file.

Just some ideas.

Thanks,

Ashley.

To unsubscribe from this group and stop receiving emails from it, send an email to play-framework-dev+unsub...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Dominik Dorn
http://dominikdorn.com | http://twitter.com/domdorn
XING: https://www.xing.com/profile/Dominik_Dorn
LINKEDIN: http://at.linkedin.com/pub/dominik-dorn/66/b42/bb1/

--

Dominik Dorn
http://dominikdorn.com | http://twitter.com/domdorn
XING: https://www.xing.com/profile/Dominik_Dorn
LINKEDIN: http://at.linkedin.com/pub/dominik-dorn/66/b42/bb1/

--

Dominik Dorn
http://dominikdorn.com | http://twitter.com/domdorn
XING: https://www.xing.com/profile/Dominik_Dorn
LINKEDIN: http://at.linkedin.com/pub/dominik-dorn/66/b42/bb1/

[Dominik Dorn]

Actually, you could just add another startup script that wraps the existing one and does this

#! /bin/bash

export APPLICATION_KEY_SECRET=`cat /path/to/secret/in/file`
./bin/myapp

then just reconfigure your docker build to run your script instead. 

To unsubscribe from this group and stop receiving emails from it, send an email to play-framework-dev+unsubscribe@googlegroups.com.

--

Dominik Dorn
https://dominikdorn.com/ | https://twitter.com/domdorn

[Ashley Aitken]

Here is alternative solution by Pawel Kaczor that solves problem with one line of code config.withSecretAt("application.key.secret"):

https://gist.github.com/pawelkaczor/8009c05eed30f67099cae70401939e55

until config perhaps /possibly is updated to allow directly:

https://github.com/typesafehub/config/issues/498#issuecomment-332200300